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Overview 



In 2007, the Texas Higher Education Coordinating Board (THECB) funded 22 colleges to 
establish developmental summer bridge programs. Aimed at providing an alternative to 
traditional developmental education, these programs involve intensive remedial instruction 
in math, reading, and/or writing and college preparation content for students entering 
college with low basic skills. In 2009, the National Center for Postsecondary Research 
(NCPR) launched an evaluation of eight developmental summer bridge programs in Texas 
(seven at community colleges and one at an open-admissions four-year university), the 
early findings of which are described in this report. 

Students who participated in the study were randomly assigned to the program group or the 
control group. Program group students participated in the developmental summer bridge 
programs, while control group students received colleges’ regular services. All 
developmental summer bridge programs had four common features: accelerated instruction 
in math, reading, and/or writing; academic support; a “college knowledge” component; and 
the opportunity for participants to receive a $400 stipend. 

The main findings of this preliminary report are: 

• All eight programs in the study were implemented with reasonable 
fidelity to the model framed by the THECB, but they varied on some 
key dimensions. 

• Program costs averaged about $1,300 per student but varied widely. 

• Program group students did not enroll in either the fall or spring 
semester at significantly different rates than control group students; 
enrollment rates were high for both groups. 

• There is evidence that the program students were more likely to pass 
college-level courses in math and writing in the fall semester following 
the summer programs. The findings also suggest that program students 
were more likely to attempt higher level reading, writing, and math 
courses compared with control group students. 




Contents 



Overview iii 

List of Exhibits vii 

Preface ix 

Acknowledgments xi 

Executive Summary ES-1 

Chapter 

1 Introduction to the Texas Developmental Summer Bridge Program 1 

The Developmental Summer Bridge Program Model 2 

Key Findings of the Study 3 

Organization of This Report 5 

2 Study Methods 7 

Site Selection 7 

Data Collection and Analysis Design 8 

Characteristics of Participating Students 1 1 

Data Sources 13 

3 The Developmental Summer Bridge Program Models 15 

The Colleges and Their Models 15 

Developmental Summer Bridge Program Features 19 

Administering Texas Developmental Summer Bridge Programs 27 

Program Updates: Summer 2010 31 

Program Costs 32 

4 The Student Experience 35 

The Control Group Student Experience 35 

The Program Group Student Experience 36 

5 Program Impacts on Educational Outcomes 43 

Key Impact Findings and Selected Outcomes 43 

Fall 2009 Semester Outcomes 46 

Spring 2010 Semester Outcomes and Cumulative Measures 50 

Subgroup Analyses 50 



V 




6 



Conclusion: A Positive Early Look 59 

The Developmental Summer Bridge Model 59 

Implications for Policy and Practice 60 

Final Follow-Up Report 60 

Appendix A: Estimating Effects 61 

References 67 



VI 




List of Exhibits 



Table 

ES.l Study Enrollment 2 

2. 1 Program Enrollment at Participating Institutions in Summer 2009 1 0 

2.2 Participant Demographics 12 

3.1 Structure of Developmental Summer Bridge Programs 21 

3.2 Academic Support at Each Participating Institution 23 

3.3 College Knowledge Components at Each Participating Institution 26 

3.4 Program Management at Each Participating Institution 28 

3.5 Program Costs 34 

4. 1 Percentage of Study Participants Enrolled in One or More College Courses 

During the Summer 2009 Term 36 

4.2 Program Enrollment and Completion 4 1 

5.1 Select Academic Outcome Measures 47 

5.2 Academic Outcomes by Gender 52 

5.3 Academic Outcomes by Program Type 55 

Box 

5.1 How to Read the Impact Tables in This Report 44 



vii 




Preface 



Each year, colleges across the nation, and open-access community colleges in 
particular, face a difficult challenge — how to improve the skills of incoming students who 
arrive underprepared for college-level coursework. Typically, colleges recommend that 
underprepared students enroll in developmental education courses; nationally, almost 60 
percent of community college students take at least one developmental education course. 
However, taking developmental courses delays the accumulation of college credits, and 
evidence suggests that the more developmental courses students must take, the less likely 
they are to ultimately earn a degree or credential. 

Developmental summer bridge programs may offer a partial solution to this 
problem. Designed to reduce the need for developmental education in college, summer 
bridge programs provide recent high school graduates with remedial instruction in reading, 
writing, or math, or some combination of these, along with an introduction to college. These 
programs, which typically run for four to six weeks during the summer, may allow students 
to advance through the developmental curriculum in a compressed time frame and ideally 
enroll in college-level courses in the fall semester. 

In addition to supporting students’ academic progress, summer bridge programs 
may also help students make the psychological and emotional adjustments involved in the 
transition from high school to college. The relationships that students develop with their 
peers and program faculty during the intensive program may strengthen their ties to college. 
Summer bridge programs may also help students become familiar with the support services 
that colleges offer and how to access them. 

Recognizing the need to increase participation and success in higher education, in 
2007 the Texas Higher Education Coordinating Board (THECB) began to encourage the 
creation of developmental summer bridge programs and other similar initiatives around the 
state. In 2009, the National Center for Postsecondary Research (NCPR) began a 
collaboration with eight Texas colleges and the THECB to study the effectiveness of the 
developmental summer bridge program model. This report describes their implementation 
and reports early findings on their impact on student outcomes. A final report, to be 
released next year, will describe students’ progression through developmental education, 
their success in college-level courses, and their persistence into and through the second year 
of college. Despite the popularity of summer bridge programs, little empirical research on 
their implementation or outcomes has previously been conducted. The research in this 
report represents an important step toward developing an understanding of how these 
programs work and what benefits they may provide for students. 
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The early results described here are modest but encouraging. While the eight 
developmental summer bridge programs examined had no effect on college enrollment 
rates, they appear to have improved student success rates in entry-level college courses in 
math and writing. Reducing the barriers to college-level coursework for underprepared 
students may increase the likelihood that these students will persist and earn a college 
credential. Developmental summer bridge programs, then, may form an important part of a 
strategy to improve completion rates at colleges in Texas and elsewhere. 



Thomas Bailey 
Director, NCPR 
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Executive Summary 



Displayed on billboards and license plates alike, “College for All Texans” is the 
unofficial motto that is promoted statewide to encourage college readiness, participation, 
and success in Texas. Policymakers, educators, and business leaders agree that Texas must 
increase rates of college participation and success to preserve the economic vitality of the 
state and to secure the future well-being of Texas residents. To address the dynamic needs 
of the growing state population, Texas launched in 2000 an ambitious statewide strategic 
plan called Closing the Gaps by 2015. One of the primary objectives of this plan is to 
increase enrollment and academic success in Texas colleges and universities. 

One component of the Closing the Gaps by 2015 initiative was the creation of 
developmental summer bridge programs — intensive summer experiences that offer eligible 
students remedial instruction in math, reading, and/or writing along with an introduction to 
college. Developmental summer bridge programs aim to reduce or eliminate the need for 
developmental courses so that more students are prepared for college-level courses in their 
first semester of college. Programs typically offer intensive, targeted coursework for four to 
five weeks over the summer, accompanied by tutoring, additional labs, and student support 
services. The integrated approach used in developmental summer bridge programs is 
thought to help ease students’ transition into college. But despite the increasing popularity 
of summer bridge programs across the country, little empirical research on their outcomes 
or impacts has been conducted. 

In 2009, the National Center for Postsecondary Research (NCPR)* launched an 
evaluation of eight developmental summer bridge programs in Texas to assess whether 
these programs reduce the need for developmental coursework and improve student 
outcomes in college. The evaluation uses an experimental design to measure the effects of 
these programs on college enrollment and success. At each college, students who consented 
to participate in the study were randomly assigned to one of two groups: a program group 
that was eligible to participate in a developmental summer bridge program and a control 
group that was eligible to receive their college’s regular services. (Random assignment 
creates two groups that are similar in both characteristics that can be measured, such as age 
or academic attainment, and those that cannot be reliably measured, such as motivation. 
This ensures that any differences in observed outcomes — called impacts — between the 

*NCPR is funded by a grant (R305A060010) from the Institute of Education Sciences, U.S. 
Department of Education. NCPR is a partnership of the Community College Research Center, Teachers 
College, Columbia University; MDRC; the Curry School of Education at the University of Virginia; and 
faculty at Harvard University. NCPR conducts studies that measure the effectiveness of programs 
designed to help students make the transition to college and master the basic skills needed to advance to a 
degree. 



ES-1 




two groups can be attributed with confidence to participation in the developmental summer 
bridge programs.) Students participated in the developmental summer bridge programs in 
summer 2009, and their academic progress is being followed through the 2010-2011 
academic year. All developmental summer bridge programs had four common features: an 
accelerated format, academic support, a “college knowledge” component, and the 
opportunity for participants to receive a $400 stipend. Eight institutions were selected for 
inclusion in this study: 

• El Paso Community College (El Paso, TX) 

• Eone Star College-CyFair (Houston, TX) 

• Eone Star College-Kingwood (Houston, TX) 

• South Texas College (McAllen, TX) 

• Texas A&M International University (Earedo, TX) 

• Palo Alto College (San Antonio, TX) 

• San Antonio College (San Antonio, TX) 

• St. Philip’s College (San Antonio, TX) 

Table ES.l shows the number of students enrolled in the study at each participating college. 



Texas Developmental Summer Bridge Programs 



Table ES.l 
Study Enrollment 



Institution 


Students in Full Sample 


El Paso Community College 


273 


Lone Star College-CyFair 


125 


Lone Star College-Kingwood 


87 


South Texas College 


138 


Texas A&M International University 


211 


Palo Alto College 


88 


San Antonio College 


152 


St. Philip’s College 


258 


Total 


1,318 
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This report is the first of two that will be published related to this research. This 
report presents early impact results from the evaluation and information on how the 
developmental summer bridge programs were implemented. It focuses on the models used, 
the range of design features incorporated, how the programs were administered, and how 
they were perceived by those involved, including college and program leaders, faculty, 
advisors, and students. A cost study of developmental summer bridge programs is also 
included. The following are the main findings of this preliminary report: 

• All eight programs in the study were implemented with reasonable 
fidelity to the model framed by the Texas Higher Education 
Coordinating Board (THECB), but they varied on some key dimensions. 

• Program costs averaged about $1,300 per student but varied widely. 

• Program group students did not enroll in either the fall or spring 
semester at significantly different rates than control group students; 
enrollment rates were high for both groups. 

• There is evidence that the program students were more likely to pass 
college-level courses in math and writing in the fall semester following 
the summer programs. The findings also suggest that program students 
were more likely to attempt higher level reading, writing, and math 
courses compared with control group students. 



Implementation of the Developmental Summer Bridge Program 

Of the eight developmental summer bridge programs included in the study, four 
were course-based, while the other four were freestanding. Course-based programs were 
essentially standard developmental courses, modified or condensed to create a shorter, more 
intensive experience. Freestanding programs were designed to provide students the 
opportunity to advance multiple skill levels by offering basic skills instruction and were not 
based on a specific course. These programs did not require students to enroll in a summer 
course and did not award any form of credit. In both course-based and freestanding 
programs, students received additional academic support, instruction in college knowledge, 
and a stipend upon successful completion. 

• All eight programs in the study were implemented with reasonable 
fidelity to the model framed by the THECB, but they varied on 
some key dimensions. 
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The goals of the summer bridge programs were primarily achieved through the 
teaching and learning that occurred in the classroom and via the various support structures. 
In most cases, faculty, tutors, and mentors worked together with the goal of facilitating 
student learning. Bundling an array of services into the programs and actively bringing 
those services to the students also featured prominently in an underlying theory of change 
for the summer bridge program model. Each of the core features — accelerated instruction 
in math, reading, and/or writing; college knowledge; academic support; and the student 
stipend — functioned together to deliver a coherent learning experience. Though there were 
many common elements across the eight programs, there were also unique features in each, 
based on the institutional contexts. 

• Program costs averaged about $1,300 per student but varied widely. 

Across the eight sites, approximately one third of costs were for staffing and just 
over one quarter for student resources. Total costs ranged from $62,633 to $296,033, which 
reflects the significant variance across sites in program enrollment, duration, and intensity. 
Across the eight sites, the average per-student cost ranged from $840 to $2,349. The 
average across all eight sites was $1,319 — an estimate of the resources needed per student 
to offer a developmental summer bridge program.^ Unsurprisingly, there is no strong 
evidence of economies of scale in terms of numbers of students enrolled; the high-value 
stipend is a constant for each student. 



Key Impact Findings 

Using data obtained from the Texas Higher Education Coordination Board and from 
the colleges that ran the summer bridge programs, we conducted several analyses of the 
overall effectiveness of the developmental summer bridge program model, comparing 
outcomes for program and control group students. Primary indicators of students’ academic 
progress included enrollment in college in the fall of 2009 and progression in 
developmental and college-level courses in math, reading, and writing. 

• Program group students did not enroll in either the fall or spring 
semester at significantly different rates than control group students; 
enrollment rates were high for both groups. 

We found that the programs did not have any impact on fall 2009 registration rates; 
that is, students in the program group registered for courses in the fall 2009 semester at a 



^Some costs may be interpreted as start-up costs and so are unlikely to be needed if the programs are 
mn in subsequent years. If these costs are amortized over three years, then the average cost of the 
programs is reduced. 
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rate that is statistically indistinguishable from the registration rate of the control group. This 
finding contradicts the hypothesis that the summer bridge programs would boost enrollment 
rates among the program group students. 

• There is evidence that the program students were more likely to 
pass college-level courses in math and writing in the fall semester 
following the summer programs. The findings also suggest that 
program students were more likely to attempt higher level reading, 
writing, and math courses compared with control group students. 

While students in the program and control groups attempted at least one math 
course at similar rates, students who participated in a developmental summer bridge 
program went on to attempt the first college-level math course at a significantly higher rate 
than students in the control group. A significantly higher percentage of program group 
students passed this first college-level math course. Program group students were also 
significantly more likely to attempt a college-level reading course and significantly less 
likely to attempt the lowest level of developmental reading. Significantly more program 
group students than control group students attempted at least one writing course and passed 
their first college-level writing course. In addition, during the 2009-2010 academic year, 
students in the program group attempted one more college-level credit than students in the 
control group. 



Looking Ahead to the Impact Findings 

Overall, the evidence catalogued in this early look at the impact of the 
developmental summer bridge programs suggests that students’ course-taking patterns are 
trending in the desired direction. In addition, these early results suggest that developmental 
summer bridge programs might help prepare students to pass introductory college-level 
math and writing courses. It is important to note that these early findings reflect student 
academic progress for only one year, and longer follow-up will provide additional evidence. 
A final report with two years of longitudinal follow-up will be released within the next year. 
We expect to learn more about students’ progression through developmental education, 
their success in college-level courses, and their persistence into and through the second year 
of college. 
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Chapter 1 



Introduction to the Texas Developmental 
Summer Bridge Program 



Displayed on billboards and license plates alike, “College for All Texans” is the 
unofficial motto that is promoted statewide to encourage college readiness, participation, 
and success in Texas. Policymakers, educators, and business leaders agree that Texas must 
increase rates of college participation and success to preserve the economic vitality of the 
state and to secure the future well-being of Texas residents. To address the dynamic needs 
of the growing state population, Texas launched in 2000 an ambitious statewide strategic 
plan called Closing the Gaps by 2015. One of the primary objectives of this plan is to 
increase enrollment and academic success in Texas colleges and universities. 

To this end, the Texas Higher Education Coordinating Board (THECB) in 2007 
funded 22 colleges to establish developmental summer bridge programs — intensive 
summer experiences that offer eligible students remedial instruction in math, reading, 
and/or writing along with an introduction to college. These programs were designed to 
provide an alternative to traditional developmental education course sequences, which may 
span several semesters, by helping students with low basic skills to build competencies 
before entering college. Summer bridge programs have become a popular strategy in Texas 
for increasing college readiness among recent high school graduates, and other two- and 
four-year colleges in the state have developed summer bridge programs independent of the 
THECB. 

Nationally, summer bridge programs have emerged as a potentially promising 
strategy for improving postsecondary success. These programs — typically held in the 
summer between high school graduation and fall matriculation in college — offer students 
accelerated, focused learning opportunities that can help them acquire the knowledge and 
skills needed for college success. But despite the increasing popularity of summer bridge 
programs across the country, little empirical research on their outcomes or impacts has been 
conducted. 

In 2009, the National Center for Postsecondary Research (NCPR) launched an 
evaluation of eight developmental summer bridge programs in Texas to assess whether 
these programs reduce the need for developmental coursework and improve student 
outcomes in college. The evaluation uses an experimental design to measure the effects of 
the programs on college enrollment and success. At each college, students who consented to 
participate in the study were randomly assigned to one of two groups: a program group that 
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was eligible to participate in a developmental summer bridge program and a control group 
that was eligible to receive their college’s regular services. 



The Developmental Summer Bridge Program Model 

The developmental summer bridge programs in this study were offered primarily to 
recent high school graduates at eight institutions of higher education — one open- 
admissions four-year university and seven community colleges — throughout the state of 
Texas in the summer of 2009.' Students attended the developmental summer bridge 
programs three to six hours daily for four to five weeks and received instruction in at least 
one area of academic need — math, reading, or writing — as well as guidance in the 
“college knowledge” needed to navigate new academic terrain. The programs were free of 
charge to the students at most sites. The developmental summer bridge programs at each 
institution in this study had four common features: accelerated instruction in math, reading, 
and/or writing; academic support; a college knowledge component; and the opportunity to 
earn a $400 stipend. All of these features except the stipend were a part of the summer 
intensive experience model framed by the THECB. The addition of the opportunity to earn 
a $400 stipend was initiated by NCPR. 

Research suggests that underprepared students who participate in summer bridge 
programs show improvement in their academic performance (Bengis, 1991). Proponents of 
summer bridge programs suggest there are several ways in which these programs may be 
more effective than traditional models of developmental education in helping students with 
below-average skill sets to build competencies and succeed in college. First, the accelerated 
instruction they provide may allow at-risk students to advance through developmental 
course sequences in a compressed time frame, ideally enabling recent high school graduates 
to enroll in college-level courses in the fall of their first year of college. Immediacy is 
important: at least one study posits that the length of time students spend in developmental 
education courses is negatively related to degree completion (Adelman, 1998). This could 
be because students in developmental education arrive at college without being adequately 
prepared for the work they will have to do there. An alternate hypothesis is that 
developmental courses themselves tend to discourage students and lead them to drop out. If 
this hypothesis is true, enrolling students in college-level courses as soon as possible after 
high school may improve students’ probability of degree attainment. 

Another reason that developmental summer bridge programs may be more effective 
than traditional developmental education sequences is that the former offer an array of 
support services to ease the transition, both academically and socially, from high school to 



'chapter 3 contains a detailed discussion of participating institutions and programmatic components. 



2 




college. Program students are required to attend tutorials and other specialized academic 
support services (e.g., writing or math labs and advising sessions). Services such as these 
might aid students in their transition to college and encourage their success in the college 
environment (Santa Rita & Bacote, 1997). Programs link students with a mentor or advisor, 
and the intensive time frame of summer bridge programs may help students form close 
relationships with their peers and faculty. Research literature on learning communities and 
career academies suggests that cohort membership is positively related to relationship 
building (Maher, 2005; Tinto, 1997). Relationships with mentors, peers, and faculty may 
strengthen students’ academic and social support networks as well as their ties to the 
institution, which may in turn promote student persistence. 

In addition, developmental summer bridge programs help participants become 
accustomed to their new environment. Students attending open-access institutions require 
social know-how to be successful in selecting courses, obtaining help, and navigating 
bureaucracies. This knowledge is often less accessible to traditionally underserved students. 
Introducing at-risk students to the college environment and actively teaching them how to 
navigate and take advantage of services may make these students more likely to persist 
(Deil-Amen & Rosenbaum, 2003; Rosenbaum, Deil-Amen, & Person, 2006). 

Finally, students in the current study who successfully completed the program were 
provided a $400 stipend. In making the stipend part of the program model, NCPR aimed to 
enable students to devote more time to their studies by reducing their need to work — 
providing another mechanism by which summer bridge programs could potentially improve 
students’ academic performance. 

All four components of the developmental summer bridge programs were designed 
to facilitate students’ success in both the program and their future courses. While all 
programs contained the features described above, each institution tailored the 
developmental summer bridge program model to meet the needs of its students and to align 
with institutional beliefs about best practices. 



Key Findings of the Study 

This report, prepared by NCPR, documents the implementation and early findings 
of the efficacy of eight developmental summer bridge programs in Texas. NCPR is a 
partnership funded by a grant (R305A060010) from the Institute of Education Sciences, 
U.S. Department of Education. NCPR includes the Community College Research Center at 
Columbia University’s Teachers College, MDRC, the Curry School of Education at the 
University of Virginia, and faculty at Harvard University. NCPR conducts studies that 



3 




measure the effectiveness of programs designed to help students make the transition to 
college and master the basic skills needed to advance to a degree. 

NCPR began its work in Texas by establishing a close, cooperative agreement with 
the THECB. A number of developmental summer bridge programs had received THECB 
funding and technical assistance for two years prior to beginning the research, resulting in a 
pool of colleges with experience in implementing these programs. Houston Endowment 
also was interested in increasing participation rates in college and became another partner in 
this study. 

Following an intensive period of initial investigation into the kinds of programs 
offered around the state, eight colleges were selected to participate. NCPR worked closely 
with these colleges to recruit students who were defined as at-risk, that is, likely to be 
placed into developmental education when they entered college. Students were then chosen 
by random assignment to participate in the limited number of slots available in the eight 
programs. Students who participated in the programs, as well as those not selected to 
participate (who were instead assigned to the control group), were asked to allow 
researchers access to their enrollment and transcript data. Access to these data permitted 
researchers to assess whether participation in a summer bridge program would affect key 
student outcomes, such as enrollment in college, need for developmental education, 
academic performance, credit accumulation, and persistence. In addition, site visits were 
made to participating colleges to learn how the summer bridge programs were designed and 
implemented. 

This report is the first of two that will be published related to this research. This 
report presents early impact results from the evaluation and information on how the 
developmental summer bridge programs were implemented. It focuses on the models used, 
the range of design features incorporated, how the programs were administered, and how 
they were perceived by those involved, including college and program leaders, faculty, 
advisors, and students. A cost analysis of the developmental summer bridge programs is 
also included. The following are the main findings of this preliminary report: 

• All eight programs in the study were implemented with reasonable 
fidelity to the model framed by the THECB, but they varied on some 
key dimensions. 

• Program costs averaged about $1,300 per student but varied widely. 

• Program group students did not enroll in either the fall or spring 
semester at significantly different rates than control group students; 
enrollment rates were high for both groups. 
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• There is evidence that the program students were more likely to pass 
college-level courses in math and writing in the fall semester following 
the summer programs. The findings also suggest that program students 
were more likely to attempt higher level reading, writing, and math 
courses compared with control group students. 



Organization of This Report 

The following chapters provide greater detail about the study. Chapter 2 details the 
study’s random assignment methodology and provides information on the sample 
characteristics. Chapter 3 provides information on program implementation, administration, 
and costs. Chapter 4 details participating students’ experiences and perceptions. Chapter 5 
discusses early impacts on selected outcomes. Chapter 6 provides a brief conclusion. A 
final follow-up report, due to be released in 2012, will provide longer follow-up on students 
and additional measures of the programs’ effects. 
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Chapter 2 



Study Methods 



This chapter provides a chronological overview of the site selection process, a 
description of NCPR’s collaboration with the THECB, and information on site visit 
activities, conducted during the summer of 2008, that facilitated final selection of sites. In 
addition, this chapter describes the random assignment process, introduces the students in 
the sample, and outlines the data sources for the study. 



Site Selection 

Criteria for the eligibility of programs to be included in the NCPR research were 
defined by the legislation that established the programs that were funded by the THECB 
(see Section 29.098, Texas Education Code). Because the THECB criteria did not require 
all programs to function the same way, program designs varied somewhat across colleges. 
For example, several colleges offered developmental summer bridge programs specializing 
solely in math, while others offered math, reading, and writing. It should be noted that five 
of the programs in this study were not funded by the THECB at the time of site selection, 
but the colleges still agreed to model their programs on the THECB criteria. 

Site selection was an extensive and multi-phased process. NCPR searched for sites 
that met a set of criteria for potential inclusion in the study, specifically those that: (1) had 
offered developmental summer bridge programs for several years; (2) were able to serve 
large numbers of students; (3) offered programs targeted to students in need of remediation; 
(4) included accelerated coursework in developmental education (math, reading, and/or 
writing), academic supports, and a college knowledge component; and (5) were able and 
willing to participate in a rigorous evaluation of the program. 

To identify possible college partners, we looked first at sites that submitted 
proposals in response to the THECB request for proposals for developmental summer 
bridge program funding in 2007 and 2008. We then conducted Internet searches for summer 
bridge programs at Texas colleges and universities and followed up with phone calls to 
program administrators to learn more about the programs and share information about the 
NCPR study. Finally, in the summer and fall of 2008 we visited the colleges that best met 
our criteria. 

NCPR initially selected seven sites to participate in the study: El Paso Community 
College, San Antonio College, Texas A&M International University, St. Philip’s College, 



7 




South Texas College, and Lone Star College campuses at CyFair and Kingwood.' To 
increase the number of students participating in the study, NCPR selected an eighth site, 
Palo Alto College, in February of 2009. Three of these sites’ programs were funded by the 
THECB; the others were funded by the colleges themselves. All sites received additional 
program funding from NCPR. 

In an effort to ensure that the programs were strong for summer 2009, NCPR 
offered technical assistance to all of the sites. One faculty expert on academic preparedness 
and remediation in community colleges worked with several colleges on creating strong 
summer bridge curricula. One institution requested that NCPR provide funds that would 
allow summer bridge instructors and tutors to work with a faculty member and expert on 
developmental mathematics education to review the curriculum and refine instructional 
practices. All of the participating institutions also attended a conference in which technical 
assistance was provided on various instructional, administrative, and recruitment issues 
related to the implementation of developmental summer bridge programs. 



Data Collection and Analysis Design 

The following provides a broad overview of the data collection plan and process 
used in the NCPR research, including sample criteria, size, recruitment, and random 
assignment procedures. 

Sample Criteria 

NCPR set a total recruitment goal of 1,400 students to ensure that the evaluation 
would be sufficiently powerful to detect relevant differences in outcomes as statistically 
significant. In other words, NCPR sought enough students in both the program and control 
groups to ensure that if the programs had a meaningful impact on students we would have 
the statistical power to know that those differences were caused by the programs and not by 
chance. The specific sample goals for each college in the study were based on several 
factors, including the capacity of the program, the number of students enrolled in the 
program in previous years, and the college’s ability to maintain program quality at scale. 
These college-level enrollment goals, which ranged from 300 students (serving 180 
students) to 75 students (serving 45 students), represented a large increase over the number 
of students served in previous summer terms for many of the institutions. Four of the sites 
met or exceeded their target numbers, two reached over 80 percent of their target numbers. 



'The Lone Star College System (LSCS) runs summer bridge programs at several of its five 
eampuses. NCPR partnered with two of the LSCS eampuses, whieh were treated throughout the study as 
distinet sites. 




and two reached over 50 percent of their target numbers. At the conclusion of the random 
assignment process, there were 1,332 students in the study (95 percent of the total target); 
this relatively small shortfall does not have any analytic implications for the study. 

Recruitment 

Recruitment was a major undertaking at all of the institutions, especially where sites 
had set high recruitment goals. For example, St. Philip’s College, which had the highest 
target, enrolled 90 students in its developmental summer bridge program in 2008; the 
recruitment target (for program and control groups combined) for summer 2009 was 300 
students — a 233 percent increase. Colleges had to scale up their recruitment efforts and 
their program offerings to meet their targets. To support these efforts, NCPR provided 
funding to the colleges, stipends to the students, and technical assistance. Institutions used 
their recruitment funds for a variety of purposes, including hiring additional recruitment 
staff, sending mass mailings to qualifying high school students, developing posters and 
brochures to distribute at high schools, and providing informational breakfasts or lunches 
for high school counselors. NCPR and the colleges believed that jobs and other stimulus- 
funded opportunities might significantly deter students from participation in the summer 
bridge programs. With NCPR funding from Houston Endowment, sites were able to offer 
students the opportunity to receive a $400 stipend; this feature was added as a core 
component of the program model. Program administrators believed that the opportunity to 
earn a $400 stipend was a major incentive for many students to join the study. 

All eligible students who applied for admission into a developmental summer 
bridge program and agreed to participate in the study at participating institutions in the 
summer of 2009 were included in the research sample. About 60 percent of the students 
were assigned into the summer bridge program group (793 students) and 40 percent to the 
control group (525 students). Table 2.1 shows the numbers of students who entered the 
study from each participating college. 

All colleges reached out to students who were likely to enroll (or were already 
registered) in the fall. In addition, some of the colleges devoted significant efforts to 
recruiting students who were undecided about college attendance in hopes that the summer 
bridge program could provide extra impetus for them to attend college in the fall. 
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Texas Developmental Summer Bridge Programs 
Table 2.1 

Program Enrollment at Participating Institutions in Summer 2009 



Institution 


Program 

(60%) 


Control 

(40%) 


Total 

Recruited 


Target 

Recruitment 


% of Target 
Recruitment 


El Paso Community College 


165 


108 


273 


233 


117 


Lone Star College-CyFair 


75 


50 


125 


125 


100 


Lone Star College-Kingwood 


52 


35 


87 


75 


116 


South Texas College 


83 


55 


138 


167 


84 


Texas A&M International University 


126 


85 


211 


167 


126 


Palo Alto College 


53 


35 


88 


167 


53 


San Antonio College 


91 


61 


152 


283 


54 


St. Philip’s College 


154 


104 


258 


300 


86 


Total 


799 


533 


1,332 


1,517 


88 



Intake and Random Assignment 

While the institutions’ program staff focused much of their energy on recruitment 
during the winter and spring of 2009, the NCPR research team created materials to help the 
sites navigate the complex student intake process. NCPR drafted intake flowcharts and 
developed individualized random assignment manuals adapted to the specific needs of each 
college and university, prepared the baseline information forms (BIF) and informed consent 
forms, and created an end-of-program survey for students. These materials were all shared 
and discussed extensively with the sites during regularly scheduled conference calls in the 
months leading up to the intake sessions. 

Subsequently, NCPR research teams scheduled on-site training days for all partner 
sites. These visits included a review of random assignment manuals, training on computers 
for entering BIF information and navigating the online random assignment program, and 
informational sessions for college administrators interested in learning more about the 
NCPR study. The institutions distributed $30 gift cards to students who completed the BIF 
and went through the random assignment process. 
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Pooled Data for Analysis 

While the programs varied in their delivery and implementation (as discussed in the 
following chapters), because all of the programs contained four common features — an 
accelerated format, a college knowledge component, academic support, and a stipend — 
NCPR pooled the data across all eight sites for analysis. Our site visits during the summer 
of 2009 allowed us to verify the presence of these four features, giving us confidence that 
the programs were substantially similar. 



Characteristics of Participating Students 

The students in the summer bridge programs were predominately Hispanic, recent 
graduates of high school, and willing to spend long hours in the summer studying to prepare 
themselves to enter college in the fall. (As one said, “I want to attend so I can take college 
classes and not remedial courses and waste money.”) Based on student baseline data 
collected at the eight colleges participating in this study, the overall demographic and 
educational characteristics were very similar for the program and control groups. 
Importantly, an omnibus test for significant differences between the two groups found that 
there were no systematic differences between program and control group students on the set 
of background characteristics collected, confirming that the random assignment process 
successfully created two comparable groups of students. Table 2.2 indicates the 
characteristics of students in the sample. 

While demographic characteristics were comparable between students in the 
program and control groups, they did vary among student populations by college. For 
example, as a result of recruitment efforts that targeted continuing education students as 
well as recent high school graduates. South Texas College had a considerably larger number 
of older students than did the other colleges. While most participating students at the other 
colleges were 17-19 years old, more than half of participating students at South Texas 
College were 19 years old or older. Additionally, approximately a third of students at South 
Texas College had children — a significantly larger proportion than at the other colleges. 

In terms of race and ethnicity, El Paso Community College, San Antonio College, 
Palo Alto College, South Texas College, and Texas A&M International University served 
mostly (more than 90 percent) Hispanic students. At both Lone Star College-Kingwood and 
Lone Star College-CyFair, fewer than half of students were Hispanic (the balance was 
comprised of White; African American; and, to a lesser extent, Asian American students). 
All institutions served more females than males, with Palo Alto College having the most 
imbalanced gender ratio at almost 3:1. 
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Texas Developmental Summer Bridge Programs 



Table 2.2 



Participant Demographics 





Program 


Control 


All 


N 


793 


525 


1,318 


Gender 


Female 


62.5% 


62.1% 


62.4% 


Male 


37.5% 


37.9% 


37.6% 


Age as of June 1, 2009 


Mean age 


18.9 


19.1 


19.0 


Age 18 or under 


84.7% 


84.8% 


84.7% 


Age 19 or 20 


11.9% 


10.1% 


11.2% 


Age 21 or over 


3.4% 


5.2% 


4.1% 


Race/ethnicity 


African American 


6.8% 


6.4% 


6.6% 


Hispanic 


84.3% 


84.2% 


84.3% 


White 


8.8% 


8.6% 


8.7% 


Other 


1.8% 


2.0% 


1.8% 


Educational aspirations 


High school 


12.8% 


11.8% 


12.4% 


Some college 


1.5% 


1.2% 


1.4% 


Certificate 


1.7% 


1.2% 


1.5% 


Associate degree 


10.1% 


11.2% 


10.5% 


Bachelor’s degree 


34.9% 


34.8% 


34.8% 


Master’s degree 


34.9% 


34.2% 


34.6% 


Doctoral or professional degree 


3.2% 


4.9% 


3.9% 


Educational attainment 


Previously attempted college credits 


23.4% 


25.5% 


24.2% 


Previously earned college credits 


21.0% 


21.4% 


21.2% 


Graduated high school 


79.5% 


80.0% 


79.7% 


2007 or earlier 


1.5% 


4.6% 


3.3% 


2008 


1.9% 


2.3% 


2.0% 


2009 


72.4% 


71.% 


71.9% 


Fall 2009 college attendance plans 


Full-time (4 courses per term or more) 


77.6% 


77.7% 


77.6% 
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Table 2.2 (continued) 





Program 


Control 


All 


Part-time (3 courses per term or less) 


22.4% 


22.3% 


22.4% 


Not planning to attend 


0.0% 


0.0% 


0.0% 


Texas Success Initiative (TSI) 


Met math TSI standards 


20.2% 


21.0% 


20.5% 


Met reading TSI standards 


18.5% 


16.4% 


17.7% 


Met writing TSI standards 


39.5% 


38.1% 


38.9% 



SOURCE: Self-reported by students on the Baseline Information Form. 



NOTES: A student’s scores on one of several placement tests (i.e., ASSET, COMPASS, ACCUPLACER, 
and THEA) determine whether the student meets the TSI minimum passing standard. A passing score on 
the TSI indicates that a student is ready to enroll in freshman-level academic coursework. While 
institutions may set higher minimum passing scores than the state does, this report relies on the cut scores 
established by Texas. There was no statistically significant difference between the program and control 
group students in any of the three areas (math, reading, or writing) at baseline. Although the total number 
of students recruited was 1,332, the total number of students in the sample with usable data who remained 
in the study was 1,318. 



All students said that they planned to attend college — either part or full time — in 
the fall of 2009, the semester immediately following enrollment into the study. Since the 
colleges utilized different placement tests (ACCUPLACER, Texas Assessment of 
Knowledge and Skills, COMPASS, ACT, etc.) to determine developmental placement, we 
were unable to capture the degree of developmental need for each student. However, we 
were able to examine student scores on the Texas Success Initiative (TSI), a college 
readiness indicator used within the state of Texas. 

Of the 1,332 students who participated in random assignment, 793 students were 
selected to participate in the developmental summer bridge programs at participating 
colleges, while 525 students were assigned to the control group. Among those selected to 
participate in the developmental summer bridge programs, a total of 689 students (86 
percent) actually enrolled. 



Data Sources 

Several data sources were used for the analyses presented in this report. First, 
students completed a questionnaire called the baseline information form (BIF) prior to the 
random assignment process — that is, before they could be influenced by the program. The 
BIF collected demographic and other background information on students. Baseline data 
were used to identify students for gender-based subgroup analyses, in which the outcomes 
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for men and women were analyzed separately and compared to determine whether 
programs had different effects for men and women. Second, the eight colleges that held 
summer bridge programs provided student-level transcript data for the sample members 
(program and control) participating in the study. These data provided a detailed look at 
sample members’ performance in college through measures such as enrollment status, 
credits attempted, and credits earned. Third, data were collected from the THECB. The 
THECB collects data on student enrollments statewide, including aggregate academic 
measures, such as the level of developmental courses attempted and passed. In addition, 
qualitative data were collected through interviews and observations conducted during two- 
day visits to each of the participating colleges during the summer of 2009. 
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Chapter 3 



The Developmental Summer Bridge Program Models 



This chapter describes the eight colleges and their summer bridge programs and 
discusses the four core components of the programs: 

• accelerated instruction in math, reading, and/or writing; 

• academic support; 

• a college knowledge component; and 

• a $400 student stipend. 

This chapter reports the findings of our site visits, during which we observed how the 
program design at each college incorporated the expected program components and looked 
for indicators of high-quality implementation. This chapter also provides an overview of the 
management of the summer bridge programs, including information on the costs of 
implementing these programs. 



The Colleges and Their Models 

The eight institutions selected for inclusion in the study are El Paso Community 
College, Lone Star College-CyFair, Lone Star College-Kingwood, South Texas College, 
Texas A&M International University, Palo Alto College, San Antonio College, and St. 
Philip’s College. Of their developmental summer bridge programs, four were course-based, 
while another four were freestanding. Course-based programs were based on standard 
developmental courses, modified or condensed to create a shorter, more intensive 
experience. Students enrolled in course-based programs also received additional academic 
supports, guidance in college knowledge, and a $400 stipend to supplement their summer 
course work. Students were typically awarded developmental education credit' (below the 
college level) for completion of these courses. Freestanding programs were designed to 
provide students the opportunity to advance multiple skill levels by offering basic skills 
instruction and were not based on a specific course. These programs did not require 
enrollment in a summer course and did not award any form of credit. A brief description of 
each college and its developmental summer bridge program is provided in the pages that 
follow. 



'Developmental education credits are awarded for financial aid eligibility purposes but cannot be 
applied toward a degree. 
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El Paso Community College 

El Paso Community College (EPCC) is an urban institution that was established in 
1969. El Paso County Community College District includes five campuses; its 
developmental summer bridge program, Project Dream, was operated at four of the 
campuses. In fall 2008, EPCC had a full-time-equivalent enrollment of 15,201 students. The 
student population was about 85 percent Hispanic, 8 percent White non-Hispanic, and 2 
percent African American. These percentages closely reflect the demographics of El Paso 
County. 



Project Dream has operated since 2006 and runs for five weeks each summer. The 
program was one of three in the study to receive funding through a grant from the THECB 
for 2009-2011. Project Dream, a freestanding program, enrolled students in coursework in 
math and reading, as well as a Dream Class that incorporated writing, cultural awareness, 
and college knowledge. Program instructors, trained in pedagogy, worked to offer 
contextualized instruction. Program students were mentored by older college students. 

Lone Star College-CyFair 

Eone Star College-CyFair, established in 2003, is the newest of five campuses of 
the Eone Star College System (ESCS), which serves the suburbs of Houston. In fall 2008, 
Eone Star College-CyFair had a full-time-equivalent enrollment of 6,709 students. The 
student population in 2008 was about 40 percent White non-Hispanic, 30 percent Hispanic, 
10 percent African American, and 10 percent Asian American. 

The Summer Bridge Program has operated since 2006 and runs for four weeks. In 
this course-based program, students participated in one developmental education course in 
math, reading, or writing. All had placed into the highest-level developmental course in the 
subject area. Other important elements of the program included the presence of tutors in 
each class, the use of MyMathLab and MyWritingEab software, college knowledge 
workshops, and individually assigned mentors for every student. 

Lone Star College-Kingwood 

Lone Star College-Kingwood, which is also part of the five-campus ESCS, was 
established in 1984. In fall 2008, Lone Star College-Kingwood had a full-time-equivalent 
enrollment of 3,820 students. The student population at Lone Star College-Kingwood in 
2008 was about 55 percent White non-Hispanic, 20 percent Hispanic, and 12 percent 
African American. 
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The college has offered the four-week Summer Bridge Program since 2006. The 
course-based program offered classes in math and writing, and students took only one 
subject. Students in the writing option had class for four hours per day, while those in math 
had class for three hours per day. Both options were for students whose placement test 
scores put them at the highest level of the developmental course sequence. 

Important elements of the program included tutors who were present in each class 
and available to provide students with extra help before and after class, four one-hour 
workshops on study skills and college knowledge, and College Connection mentors with 
structured time to interact with students. 

South Texas College 

South Texas College, founded in 1993, is a community college that recently began 
to offer baccalaureate degrees. It is located in McAllen, Texas, near the border of Mexico. 
In fall 2008, South Texas had a full-time-equivalent enrollment of 12,644 students. The 
student population at South Texas is primarily Hispanic (about 95 percent). Interestingly, 
the Hispanic student population of South Texas College is higher than the proportion of 
Hispanics in the city of McAllen. 

The summer bridge program, called Jumpstart, was established in 2007 (the 
program did not run in 2008). A freestanding program, Jumpstart runs for four weeks and 
focuses only on developmental math instruction. Significant elements of the program 
included weekly college knowledge workshops, use of MyMathLab to supplement 
instruction, tutors in the classrooms and labs, and daily journal writing to informally 
develop writing skills. Students in the math courses were grouped without regard for the 
level of remediation needed. 

Texas A&M International University 

Texas A&M International University (TAMIU) is a four-year institution that first 
started to accept students in 1970 and became a campus of the Texas A&M University 
System in 1989. It is the only university in the NCPR study and one of three institutions in 
the study to receive funding through a grant from the THECB. Located in Laredo on the 
border of Mexico, TAMIU has a heavily Hispanic student population (over 90 percent). In 
fall 2008, TAMIU had a full-time-equivalent enrollment of 4,222 students. 

The freestanding summer bridge program at TAMIU started in 2007. In previous 
summers, the program offered English and math, but for the summer 2009 program, the 
staff decided to focus solely on developmental math instruction with all levels mixed 
together in each section. The program was called Intensive College Math Prep. In addition 
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to the intensive, accelerated math instruction, other important elements of the program were 
tutor-administered math lab hours, weekly meetings with older-student mentors, and weekly 
college knowledge presentations. The program ran for five weeks. 

Palo Alto College 

Palo Alto College, a member of the Alamo Community College District (ACCD) in 
San Antonio, was established in 1983. In fall 2008, the college had a full-time-equivalent 
enrollment of 4,486 students. The student population in 2008 was about 66 percent 
Hispanic, 30 percent White non-Hispanic, and 2 percent African American. The 
demographics of Palo Alto College closely resemble those of greater San Antonio, where 
59 percent of the population are Hispanic or Latino of any race. 

In the past, Palo Alto College ran summer bridge programs that focused on 
SAT/ACT preparation. In the summer of 2009, Palo Alto College started the Early Start 
program, which offered only developmental math in a course-based format. Students 
attended class for four weeks. Distinctive features of the program included tutors in each 
class, provision of breakfast and lunch to all students, a cohort model, and a student 
development course. 

San Antonio College 

San Antonio College, also a member of ACCD, was established in 1925 and in fall 
2008 had a full-time-equivalent enrollment of 12,226 students. The student population is 
about 48 percent Hispanic, 42 percent White non-Hispanic, and 5 percent African 
American. The summer bridge program at San Antonio College, established in 1999, is 
called Senior Summer. 

Senior Summer allows students to enroll in developmental math, reading, and 
writing as well as college-level courses. Participating students chose two courses from the 
regular college summer offerings. The program ran three and a half hours per day for five 
weeks, with an additional hour two days each week. The program staff emphasized that 
they recruit students who are often overlooked as “college material.” Important elements of 
Senior Summer included enrollment in actual developmental-level courses, a pared-down 
version of a student development course, assistance with financial aid forms, and fostering 
college-going attitudes among students. 

San Antonio College had difficulty recruiting students for the 2009 summer 
program. In previous years, program staff had worked closely with individual students 
during the recruitment process to provide assistance in completing financial aid forms, 
which is a key part of this course-based program because students pay tuition. Because of 
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the random assignment design, it became more complicated to work effectively with 
individual students. 

St. Philip’s College 

St. Philip’s College, another ACCD member in San Antonio, is a historically Black 
college and Hispanic-serving institution that was founded in 1898 and became a public two- 
year institution in 1942. In fall 2008, St. Philip’s College had a full-time-equivalent 
enrollment of 5,533 students. The student population is about 47 percent Hispanic, 34 
percent White non-Hispanic, and 16 percent African American. The freestanding summer 
bridge program is called the Fresh X Summer Program. 

The Fresh X program provides developmental education instruction in math, 
reading, and writing. Students took courses in one, two, or all three developmental subject 
areas based on remediation needs as indicated by ACCUPLACER^ test scores. The contact 
hours of the program varied depending on the number of courses taken. The program ran 
five days per week for four weeks. Fresh X was marketed as an ACCUPLACER refresher 
course, and the goal of the program was to improve students’ scores on the test. The 
instruction was heavily focused on passing the test as a result. Other important elements of 
the Fresh X program included support from the math, writing, and reading labs; a credit- 
bearing student development course; and use of adaptive learning software to supplement 
math instruction. 



Developmental Summer Bridge Program Features 

The goals of each summer bridge program were primarily achieved through the 
teaching and learning that occurred in the classroom and the various support structures. In 
most cases, faculty, tutors, and mentors worked together with the goal of facilitating student 
learning. Bundling an array of services into the programs and actively bringing those 
services to the students — or being “intrusive,” as one program administrator described it 
— features prominently in the underlying theory of change for the summer bridge program 
model. Teaching and learning at each site were directly influenced by the structure of the 
summer bridge program. Each of the core features — accelerated instruction in math, 
reading, or writing; academic support; the college knowledge component; and the student 
stipend — functioned together to deliver a coherent learning experience. Though there were 
many common elements across the eight programs, each program also had unique features 
to accommodate the institution’s student population or, in some cases, to fit with the 



WCCUPLACER is an adaptive placement test developed by the College Board. 
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academic calendar or institutional culture. This section describes each of the four core 
program features in detail. 

Accelerated Instruction in Math, Reading, and/or Writing 

All sites offered developmental education instmction in an accelerated format, with 
some degree of contextualized learning and active learning. Formats, schedules, and content 
differed from site to site and were influenced by the subject areas taught, as well as by 
program goals. South Texas College, TAMIU, and Palo Alto College offered only math; 
Lone Star College-Kingwood offered math and writing; and St. Philip’s College, EPCC, 
Lone Star College-CyFair, and San Antonio College offered a combination of math, 
reading, and writing. A few programs offered one developmental subject (e.g., math) but 
taught students of different developmental levels together in a freestanding program. 

The length and intensity of the developmental summer bridge programs also varied, 
as shown in Table 3.1. The minimum time required to complete a program was about 35 
hours for a student taking only one subject at St. Philip’s College; a large majority of the 
college’s students, though, took two classes for over 50 hours of program time. Conversely, 
students enrolled at EPCC or TAMIU spent approximately 100 hours on campus. These 
hours represent required time at the college and do not include hours spent on homework or 
group project assignments. 

Instructors appreciated the flexibility offered by the freestanding programs. 
Freestanding programs could be structured in the ways that program coordinators and 
faculty members thought best, without needing to follow pre-established curriculum 
guidelines. As one faculty member from EPCC stated, “I think what makes this program 
neat is that it is different from a [regular developmental] class because you are allowed to 
kind of step out of the box and be a little more creative with what you are doing, as opposed 
to having to follow the syllabus.” A student at EPCC saw this flexibility as a way to smooth 
the passage between high school and college and commented, “It’s not a hardcore college 
class, but not high school either. ... It makes the transition easier.” 

Student assessment practices also differed between the course-based and 
freestanding programs. As with a standard developmental course, students enrolled in 
course-based programs were graded based on performance; students who earned a C or 
better earned developmental credit and were allowed to advance to the next course in the 
sequence. In contrast, students in freestanding programs were not given traditional grades. 
Rather, they worked toward retaking and passing the institution’s placement test 
(COMPASS, ACCUPEACER, etc.) at the end of the program. All programs were designed 
to help developmental students advance at least one level of proficiency during the program. 
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Texas Developmental Summer Bridge Programs 



Table 3,1 

Structure of Developmental Summer Bridge Programs 



Institution 


Hours 


Weeks 


Course- 

Based 


Levels Below 
College Level 


Student 

Ability 

Levels 


Subjects 

Offered 


El Paso 












Math, 


Community 


100 


5 


No 


All 


Mixed 


Reading, and 


College 












Writing 


Lone Star 
College-CyFair 












Math, 


67 


4 


Yes 


1 


Not Mixed 


Reading, and 
Writing 


Lone Star 

College- 

Kingwood 


64 (Math) 
52 (English) 


4 


Yes 


1 


Not Mixed 


Math and 
Writing 


South Texas 
College 


80-100 


4 


No 


All 


Mixed 


Math 


Texas A&M 
International 
University 


100 


5 


No 


All 


Mixed 


Math 


Palo Alto College 


60-76 


4 


Yes 


2 and 3 
(lowest) 


Not Mixed 


Math 


San Antonio 
College 












Math, 


Approx. 97 


5 


Yes 


All 


Not Mixed 


Reading, and 
Writing 


St. Philip’s 
College 


Approx. 

35-95 


4 


Yes 


All 


Mixed 


Math, 

Reading, and 
Writing 



Program administrators generally believed that the accelerated format offered 
students a chance to quickly increase their readiness for college. A number of faculty 
members concurred, believing that there were clear instructional advantages to focusing on 
one or two subject areas exclusively for a short period of time. However, others found this 
format to be pedagogically challenging. Some faculty remarked that a summer program is 
not ideal and believed that lower achieving students stmggled with the intensity of the 
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work. One math faculty member stated, “A lot of these students have tested in 0300, which 
is integers ... middle school math! They need more math, not less.” In addition, some 
instructors expressed frustration with teaching the students all of the necessary course 
material within a shortened time frame. 

Faculty in the course-based programs taught the standard developmental course at 
an accelerated pace typical of summer courses in general. The summer bridge programs did 
not provide modified software, textbooks, or materials to accommodate the shortened time 
frame. If students successfully completed the coursework, they became eligible to advance 
to the next level without retesting. Faculty in the freestanding programs expressed both 
positive and negative views on teaching students of multiple ability levels in a single 
classroom, especially about the benefits and drawbacks of this arrangement for the lowest 
level students. A South Texas College math faculty member commented, “It doesn’t affect 
me. ... I start at the very bottom in any case. Everyone makes progress together. This seems 
to work well.” However, another South Texas College math instructor feared that this 
arrangement would hurt the lowest level students’ chances of passing. Institutions utilized 
different methods of addressing this challenge. For example, EPCC had mixed-ability math 
and writing classes for most of the day, followed by groupings according to reading ability 
for a portion of instructional time. This approach allowed the lower ability students to work 
on basic reading skills while their higher ability peers discussed a book they were reading 
outside of class. 

Academic Support 

Academic support was an important part of the developmental summer bridge 
programs, not only because it supported the instructional component but also because it 
helped to integrate students into the life of the program and the college. This support 
generally took the form of tutoring, mentoring, or access to learning labs and computer- 
based programs (see Table 3.2). The math programs generally utilized some form of 
mandatory computer-aided instruction and independent practice. Some of the reading and 
writing programs did so as well. Three programs provided students with a subscription to 
MyMathLab, and two programs utilized Assessment and Eeaming in Knowledge Spaces 
(AEEKS). Students spent part of their day on campus working through individualized 
lessons targeted to their specific needs, and teachers were able to keep track of the students’ 
progress virtually. If students had Internet access outside of the school, they could log on 
remotely, but this was not required by any of the programs. 

South Texas College took this approach one step further and utilized MyMathTest 
(created by the same company as MyMathLab) as a substitute for their textbook. Although 
this worked well for some students, others — especially those without Internet access at 
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home — expressed a desire to have a physical textbook so that they could work 
independently at home. One student without home Internet access said she was “try[ing] to 
write as much notes as possible because we don’t have the book at home.” 



Texas Developmental Summer Bridge Programs 
Table 3.2 

Academic Support at Each Participating Institution 



Institution 


Academic Support 


El Paso Community College 


Mentoring program; mentor assigned to each 
class; structured time with mentors 


Lone Star College-CyFair 


Mentoring; tutor assigned to each class 4 hours 
per week 


Lone Star College-Kingwood 


Tutors present for entire time of each class; 
structured time with mentors 


South Texas College 


3 tutors and an estimated 3^ hours per week of 
lab time 


Texas A&M International University 


Involvement with mentors, tutors, academic 
support advisor, and program director 


Palo Alto College 


Tutors in class; mandatory daily tutoring 
sessions 


San Antonio College 


Tutoring and lab time available on a voluntary 
basis 


St. Philip’s College 


Use of labs and tutoring 



San Antonio College and St. Philip’s College were the only two colleges that did 
not provide tutors as part of the summer bridge program; however, San Antonio College 
encouraged math students to spend 1 0 hours per week in the math lab, where tutoring was 
available. The other six programs provided various forms of tutoring. For example. Lone 
Star College-CyFair assigned a tutor to each class for four hours per week. Lone Star 
College-Kingwood had a tutor in every classroom for the entire class period. South Texas 
College had three tutors roaming between five classes. TAMIU had two or three tutors 
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present in the classroom as well as in each computer lab while the students were completing 
their lab work. A math faculty member at Palo Alto College, in response to a question about 
the single most important element of the program, replied, “I think it is the tutoring time. 
[Students] are forced to do their homework. If not forced, some would not do their 
homework. I think it enhances the course. That is going to make it a success.” Many 
instructors stated that requiring students to practice newly acquired skills at a set time 
during which help was accessible was important to program success. 

Lone Star College-Kingwood specifically chose tutors who could relate to the 
students (e.g., tutors who were approximately the same age as students, or tutors who were 
present or former Lone Star College-Kingwood students) rather than professional tutors 
with master’s degrees in an effort to increase students’ comfort. These tutors, present for the 
entirety of every class, modeled academic skills, such as note-taking. The tutors felt that 
this was particularly effective because students could compare their notes to those taken by 
the tutors. By minimizing the distinction between student and tutor, the tutors often became 
academic models or mentors to the students. 

Structured mentoring was utilized as an explicit program component in four of the 
summer bridge programs (i.e., EPCC, TAMIU, Lone Star College-CyFair, and Lone Star 
College-Kingwood). Mentoring was one of the methods that colleges used to connect and 
engage students with the college and its resources. The two Lone Star Colleges assigned 
their College Connection^ advisors as mentors, since these individuals had extensive 
experience in high schools. These institutions did not have a structured training program or 
predetermined topics for the mentors to cover with students. Consequently, the quality and 
intensity of the mentoring varied according to the individual mentor. Faculty at Fone Star 
College-CyFair reported not knowing who individual students’ mentors were; thus, 
mentors could not be contacted when a student was having difficulty. TAMIU and EPCC 
had strong mentoring programs with specific meeting times and topics identified in advance 
in consultation with program supervisors. This structure appears to have maximized the 
effectiveness of the time students spent in advising sessions. 

College Knowledge Component 

In this paper, college knowledge instruction is defined as the provision of 
information about localized college contexts (e.g., tours, available student services, etc.), 
college expectations (study skills, faculty norms, time management, etc.), college planning 
(course-taking, plans for transfer), aligning educational goals with career plans, and paying 
for college. All programs in the study provided some explicit means to help students gain 

^College Connection advisors are used throughout the state to assist high school students in the 
transition to college. 
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college knowledge, but the specific content and delivery method varied (see Table 3.3). 
Content was formally delivered in one of two main ways. Three programs (Palo Alto 
College, San Antonio College, and St. Philip’s College) utilized a modified version of a 
student development (or “student success”) course of the type generally offered to new 
college students. These courses were often taught by college advisors and had a designated 
time slot in the day’s schedule. The other five programs (i.e.. South Texas College, Lone 
Star College-Kingwood, Lone Star College-CyFair, TAMIU, and EPCC) provided 
presentations focused on different topics (e.g., financial aid, career assessment, etc.). 
Colleges generally offered four or five one-hour presentations, once per week. These 
presentations were not generally integrated into the curriculum and were taught by either 
advisors or mentors. The mentors at the two Lone Star Colleges conducted the four 
presentations on college knowledge, allowing for greater continuity between the overall 
program goals and the college knowledge component than in some of the other programs. 
According to the end-of-program survey, students appeared to value the abbreviated student 
success courses more than the presentations, which they sometimes found to be 
disconnected from the rest of the program. 

College knowledge can be divided into social and academic components. In the 
developmental summer bridge programs, academic knowledge was stressed more than 
social knowledge. The programs covered a variety of topics in academic college 
knowledge, including study and test-taking strategies, time management, career assessment, 
learning styles, tours of the campus, introduction to college resources, financial aid, and 
course or degree plans. The information needed to make the social transition to college was 
covered to some degree in all summer bridge programs. Students in the programs received 
instruction in social college knowledge topics, such as personal financial responsibility, 
motivation, behavioral expectations, and techniques for dealing with stress. 

Although the colleges provided formal courses and sessions, college knowledge was 
also imparted informally. Many faculty members and tutors informally integrated 
information about how to navigate college into the classroom or hallway conversations. For 
example, faculty members would comment on the differences, both academic and social, 
between high school and college. As one TAMIU faculty member noted, “[We] tell them 
from day one that they are college students and in college.” Explicit statements like these 
helped to communicate to students that they were in a more demanding academic setting 
and were expected to act accordingly. Tutors gave tips on study skills and how to anticipate 
what a faculty member might ask on a test. TAMIU and EPCC mentors and students spent 
much of their scheduled time together focused on college knowledge topics, such as 
advising and registration. 
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Texas Developmental Summer Bridge Programs 



Table 3.3 

College Knowledge Components at Each Participating Institution 



Institution 


College Knowledge Components 


El Paso Community College 


Dream Class; college knowledge presentations 


Lone Star College-CyFair 


Career assessment; use of Productivity Environmental Preference Survey 
(PEPS) to identify learning styles; one-on-one orientation given by mentors 


Lone Star College-Kingwood 


4 workshops run by mentors; individual meetings with mentors 


South Texas College 


4 presentations on college success 


Texas A&M International University 


4 presentations on college success; assistance with applications and 
financial aid; regular individual meetings with mentors 


Palo Alto College 


Student development course (1 credit); assistance with financial aid; use of 
Que Paso? text 


San Antonio College 


Student success course (1 credit) 


St. Philip’s College 


Student development course (1 credit) 



Stipend 

Students had the opportunity to earn up to $400 upon successful completion of the 
developmental summer bridge program. This incentive was built into the program design to 
boost recruitment and increase the likelihood that students would complete the program 
once enrolled. Our plan called for the first payment of $150 to be disbursed to students at 
the end of the first week of the program. The balance of $250 was to be paid out to students 
who successfully completed the program. Each college defined successful completion on its 
own. For example, Lone Star College-Kingwood defined successful completion as 
receiving a grade of C or better and having missed no more than three days of class during 
the program. 

Administrators believed that the $400 stipend was a powerful recruitment tool, and 
student feedback confirmed this. Some students stated that the funds offset the need to work 
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more hours, while others perceived the stipend as a kind of bonus. In addition, both 
program administrators and students saw the stipend as contributing to program completion. 
One student indicated that knowing the stipend was there helped him to get up and make it 
to class in the morning. 

Though all students who qualified were paid $400, the initial payment was almost 
universally delivered a week or two late due to various bureaucratic difficulties. The delay 
resulted in a number of student complaints and inquiries about the stipend. When asked 
about the effect of the stipend, one student pointed out that “we haven’t received any of it 
yet — but we’re still liking [the program], so that’s good.’’ Some students admitted that 
they experienced hardship because they did not receive the funds on time, but attrition from 
the program was not significant. Overall, students and administrators reported that the 
stipend helped to draw students to the study and induced eligible students to attend and 
complete the summer bridge program. 



Administering Texas Deveiopmentai Summer Bridge Programs 

This section provides an overview of the administration of the developmental 
summer bridge programs. It outlines the organizational features of the programs and how 
they relate to the colleges, as well as the types of administrative support offered by each of 
the colleges. Additionally, this section describes the recruitment and hiring of key staff, 
including faculty, mentors, and tutors. It then examines the types of support that college 
faculty offered to the program. This section concludes with a program cost analysis. 

Organizational Location Within the College 

In general, the developmental summer bridge programs were not part of the 
departmental education structure within colleges (see Table 3.4). Because these were 
special programs serving students before they officially entered college and because five 
programs included instruction in more than one subject area, nonacademic staff generally 
held leadership positions. The only programs run by faculty were the ones at South Texas 
College and TAMIU, although EPCC’s program was codirected by a faculty member. It is 
possible that the presence of nonacademic staff in leadership roles resulted in a greater 
emphasis on program structure and student support than on instructional innovation. 
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Texas Developmental Summer Bridge Programs 



Table 3,4 

Program Management at Each Participating Institution 



Institution 


Program Management 


El Paso Community College 


Codirected by director of Student Success and a faculty member 


Lone Star College-CyFair 


Run by director of Outreach and Retention 


Lone Star College-Kingwood 


Run by dean of Student Development with a part-time coordinator 


South Texas College 


Run by a math (non-developmental education) faculty member; overseen 
by associate dean of Community Engagement and Workforce Development 


Texas A&M International University 


Run by a developmental English faculty member; overseen by the 
executive director of Programs for Academic Support and Enrichment 


Palo Alto College 


Run by Gateway to College coordinator; overseen by vice president of 
Student Affairs 


San Antonio College 


Run by off-site coordinator and student support specialist of Evening, 
Weekend and Distance Education (EWDE); overseen by dean of EWDE 


St. Philip’s College 


Run by three members of the First Year Experience Center; overseen by 
the center’s dean 



Internal College Support for Programs 

Leaders at participating colleges expressed great concern about both the number of 
students who test into developmental education and the need to broaden access to college. 
College leaders had a particular interest, expressed in a variety of ways, in meeting the 
needs of the local community and increasing college attendance and success rates among 
Hispanic students. The dean at one college was a former migrant farm worker with a strong 
personal commitment to improving opportunities for the next generation. In other cases, the 
desire to increase access to and success in college was closely tied to community 
development goals. 
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Program Planning, Implementation, and Oversight 

The management of summer bridge programs was typically labor-intensive, and 
participation in the NCPR research added to the amount of work involved. Most notably, 
identifying and recruiting student participants posed a challenge, and significant efforts 
were made to promote the programs at local high schools. The need to accept applications 
from more students than any given program could accommodate (to permit the use of an 
experimental research design) only added to the pressure on program staff. However, NCPR 
supported the sites in this effort and hired a consultant to work with the sites based in 
Houston (Lone Star College-CyFair and Lone Star College-Kingwood) and San Antonio 
(Palo Alto College, San Antonio College, and St. Philip’s College). Together, NCPR and 
the consultant offered sites ideas to boost recruitment, help in developing promotional 
materials, and assistance with the random assignment process. NCPR also encouraged the 
sites to forge close ties with high schools and adopt an active approach to recruitment, 
rather than simply marketing the programs and expecting students to apply. It seems likely 
that many more students learned of the summer bridge programs than in the past and that 
more disadvantaged students were reached. 

Apart from the student recmitment effort, management of these programs was 
handled without major difficulties. The college units in charge were experienced in the 
management of these initiatives and similar initiatives, and all had run summer bridge 
programs for at least two years prior to the summer of 2009. 

Recruitment Linkages with Other Organizations 

Recruitment efforts for developmental summer bridge programs were enhanced in 
settings in which there was a strong connection between the college and its feeder high 
schools. As noted above, recruitment was conducted primarily at local high schools. Strong 
relationships with high school counselors allowed the colleges to reach students and market 
the programs. Counselors are in a good position to refer the students who they feel can 
benefit the most from the summer bridge programs. In fact, 40 percent of the developmental 
summer bridge participants indicated that they had learned about the programs from their 
high school counselors. Many of the colleges in this study were recipients of College 
Connection'^ grants that enabled them to regularly send staff to the high schools. The 
College Connection staff were typically trusted by students and high school guidance 



"^College Connection, funded by the THECB, supports the state’s Closing the Gaps by 2015 initiative 
by (1) building awareness among high school seniors of the importance of a college education; (2) 
increasing rates of participation in higher education by providing students with assistance at multiple 
stages of the college entry process, including admission, financial aid application, and enrollment; and (3) 
increasing the number of students who successfully complete the transfer application process to the four- 
year institutional partner. 
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counselors and were already interacting with students on a regular basis. Thus, they were 
well positioned to present the summer bridge programs at college fairs, make presentations 
in classrooms, and pave the way for regular summer bridge program staff to contact 
students. 

In a few cases, linkages between the college and community organizations were 
also important. Additional recruitment was possible in these settings mainly via posters and 
flyers. 



Faculty Recruitment and Preparation 

The process of hiring of faculty for the developmental summer bridge programs 
varied considerably between colleges. Palo Alto College and Lone Star College-CyFair 
went through a careful process to hire highly skilled, engaged faculty for the developmental 
summer bridge programs. At these colleges, program leaders expressed a desire to find 
faculty who would create a comfortable learning environment for students. Additionally, 
administrators looked for professors who were fun, lively, and technologically savvy. As 
one administrator from Palo Alto College stated, “[We were not looking for] lecture 
instructors or multiple-choice instructors.” 

The remaining colleges did not handpick their faculty. Selection to participate in 
developmental summer bridge programs generally occurred at the last minute, leaving little 
to no time for preparation; in some cases, faculty positions were given to any faculty 
member who wanted to teach during the summer. 

Mentor Recruitment and Preparation 

Mentors were involved in the programs at four of the eight participating colleges. 
Mentors were generally selected based on their ability to relate their own personal 
educational experiences and college knowledge to the participants’ experiences. For 
example, at EPCC, one mentor stated, “It’s about being a peer to them. It allows you to 
connect to them on a more personal level so that you can understand their learning styles 
better than a professor would.” The mentors selected often had previous experience with 
developmental summer bridge programs, either as mentors, students, or recruiters. 

At least two participating institutions (EPCC and TAMIU) offered specialized 
training to mentors prior to the start of their developmental summer bridge programs. 
However, in the case of EPCC, several of the mentors had not yet been hired at the time of 
training. 
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Tutor Recruitment and Preparation 



The hiring of tutors, like the hiring of faculty and mentors, also varied across 
colleges. The program coordinators generally vetted all tutors. The selection of tutors was 
similar to the selection of mentors: primary consideration was given to applicants who were 
roughly the same age as participating students and who possessed strong subject matter 
knowledge in math, reading, or writing. As a representative at Lone Star College- 
Kingwood noted, “Proximity in age increases comfort level.” In some cases, tutors were 
required to have an understanding of algebra in addition to other kinds of specialized 
knowledge. Like mentors, tutors were expected to informally help participants with 
adjusting to life in college. 

Faculty Support 

Support from involved faculty for the developmental summer bridge programs 
varied more than administrative support. Faculty members who were selected based on their 
fit for developmental summer bridge programs were generally enthusiastic about the 
programs — yet there were some exceptions. For instance, faculty members at one college 
questioned the value of the developmental summer bridge program. Nonetheless, the 
majority of faculty were characterized by program administrators as gracious, supportive, 
invested, and involved. Many faculty members, including some who were hired at the last 
minute, demonstrated strong support for the programs’ aims. 

Summary 

The administration and organization of the developmental summer bridge programs 
varied across colleges, but our qualitative research suggests that the program administrators 
implemented these programs in ways that were largely consistent with a program model 
framed by the THECB and complying with the expectation of NCPR for the inclusion of 
specific program elements. 



Program Updates: Summer 2010 

In 2010, five of the eight colleges ran developmental summer bridge programs 
similar to those they offered in 2009. At each of these sites, program size was considerably 
smaller, in large part because the colleges lacked NCPR funding and because institutional 
budget cuts constrained program capacity. Recruitment also presented challenges for the 
colleges, since none of the programs were able to offer the $400 stipend provided in 2009. 
Two colleges offered a stipend of up to $200, one college offered tuition reimbursement. 
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and another college provided books and calculators for regular attendance, but the financial 
incentive to students was in no way comparable to that of 2009. 

Colleges did learn a variety of recruitment techniques by way of their participation 
in the 2009 study. In particular, administrators remarked that direct contact with the high 
schools is integral to any college recruitment effort. As a result. College Connection 
advisors played a key role in recruiting 2010 summer program participants. One program 
administrator believed that the reputational success of the 2009 program aided in recruiting 
participants for the 2010 program. 

While most of the colleges did not change their instructional offerings in significant 
ways, there were some notable changes. One college added a developmental reading course 
to its program, while another institution revised all reading and writing curricula within its 
freestanding program. A few colleges changed their math software programs or provided 
more training for mentors and tutors. One college also provided faculty development in 
2010 for the first time. 

The three colleges that did not offer programs in 20 1 0 cited diminished resources as 
the primary reason for eliminating them. Each of the colleges decided to devote their 
limited resources and primary attention to strengthening other college readiness or 
developmental student success programs (e.g., career academies, success courses for 
students in developmental math, success courses for students on academic probation, etc.). 



Program Costs 

NCPR performed an analysis of the cost of the developmental summer bridge 
programs in order to be able to eventually perform a cost-benefit or cost-effectiveness 
analysis. However, for the purposes of this report, it is useful to understand the costs 
associated with running programs of this kind. The costs of the developmental summer 
bridge programs were calculated using the ingredients method (Levin & McEwan, 2001). 
Information on costs was collected from responses by program staff to a standard 
questionnaire devised for the developmental summer bridge program.^ The main costs were 
for staffing (instructors, managerial staff, and administrative staff) and for the student 
stipends (approximately $400 per participating student). None of the colleges applied a 
direct cost-recovery formula for overheads to the program, so a uniform rate of 30 percent 



^The questionnaire included 10 components, with information collected on participation rates; 
program duration; teaching staff; non-teaching, administrative, and recruitment staff; in-kind resources; 
course preparation; materials, facilities, and overheads costs; funding sources; tuition and fees; and any 
other expenses. Wage estimates include fringe benefits (if incurred). The questionnaire form is available 
on request. 
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was applied on all direct costs.^ For simplification, we divided the cost ingredients into four 
groupings: staffing; other costs; student resources; and overhead. All figures are presented 
in 2009 dollars. 



This report contains information about the developmental summer bridge programs 
and data on our four cost ingredients across each of the developmental summer bridge sites. 
From these we can calculate the total cost and the cost per admitted student (not all of 
whom actually entered or completed the programs). Table 3.5 summarizes the total costs for 
each of the eight sites and the average across all eight sites. Across the eight sites, 
approximately one third of costs were for staffing and just over one quarter for student 
resources. Total costs ranged from $62,633 to $296,033 per college, which reflects the 
significant variance across sites in program enrollment, duration, and intensity. In addition. 
Table 3.5 shows the average overall cost per participant. Across the eight sites, the average 
per-student cost ranged from $840 to $2,349. The average per-student cost across all eight 
sites was $1,319; this is the expected resource requirement needed per student to offer a 

-j 

developmental summer bridge program. Unsurprisingly, there is no strong evidence of 
economies of scale in terms of numbers of students enrolled; the high-value stipend is a 
constant for each student. 



*The 30 percent overhead rate is based on review of overhead rates applied at colleges across the 
United States. We test for sensitivity to the overhead rate below. 

^Some costs may be interpreted as start-up costs and so are unlikely to be needed if the programs are 
mn in subsequent years. If these costs are amortized over three years, then the average cost of the 
programs is reduced. 
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Texas Developmental Summer Bridge Programs 



Table 3.5 
Program Costs 





El Paso 
Community 
College 


Lone Star 
College- 
CyFair 


Lone Star 
College- 
Kingwood 


South Texas 
College 


Texas A&M 
International 
University 


Palo Alto 
College 


San Antonio 
College 


St. Philip’s 
College 


Average 
Across Sites 


Students admitted to 
program 

Cost per student 


165 


75 


52 


83 


126 


53 


91 


154 




Staffing^ 


$504 


$222 


$393 


$418 


$726 


$278 


$587 


$168 


$426 


Other costs'’ 


$68 


$183 


$425 


$165 


$570 


$365 


$97 


$175 


$235 


Student resources” 


$414 


$238 


$424 


$451 


$512 


$357 


($38) 


$373 


$354 


Overhead 


$296 


$193 


$373 


$310 


$542 


$300 


$194 


$215 


$304 


Cost per student 
admitted 


$1,282 


$835 


$1,615 


$1,343 


$2,349 


$1,299 


$840 


$930 


$1,319 


Total cost 


$211,512 


$62,633 


$83,963 


$111,463 


$296,033 


$68,853 


$76,458 


$143,218 


$123,160 



NOTES: “Includes instruction, coordination, and recruitment. '’Includes program preparation, facilities, and learning resources. “Includes stipends net of fees and meals. 



Chapter 4 



The Student Experience 



In this chapter, we provide information on the students — both those who 
participated in developmental summer bridge programs and those who did not. We first 
offer insights into the experiences of students who were not among the 60 percent of our 
sample admitted to developmental summer bridge programs in the summer of 2009. Then, 
we discuss the characteristics, experiences, and outcomes of students who participated in 
the program. With our qualitative inquiry, we sought to understand patterns of program 
participation and to ascertain if programs met the perceived needs of student participants. 



The Control Group Student Experience 

While students in the control group were not tracked as closely during the summer 
of 2009 as their counterparts in the program group, student baseline data and summer 
course enrollment data from the eight colleges participating in this study make it possible to 
provide an approximate account of their summer experience. Their experience represents 
the “business-as-usual” comparison for the evaluation, indicating what students would do if 
not offered the opportunity to enroll in a developmental summer bridge program. Based on 
student data collected during the intake and random assignment process, 70 percent of the 
students in the control group indicated that if they were not accepted into a developmental 
summer bridge program, they would be likely to work. Twenty-two percent of control 
group students indicated they would enroll in a different educational program at the college 
where they applied for admission to a developmental summer bridge program. Eight 
percent of control group students suggested that alternative plans for the summer of 2009 
included enrolling in an educational program other than those offered at their college of 
random assignment. Other students responded that they would attend summer school for 
classes (3 percent) or engage in studying on their own or with others (5 percent). 

Summer enrollment data from the eight participating colleges, summarized in Table 
4. 1 , show that 1 6 percent of students in the control group actually enrolled in a course at the 
college where they were applied for admission to a summer bridge program sometime 
during the summer of 2009. Generally, summer course enrollment rates were higher among 
control group students at colleges that offered course-based developmental summer bridge 
programs (i.e.. Lone Star College-CyFair, Lone Star College-Kingwood, Palo Alto 
College, and San Antonio College) than at those that offered freestanding developmental 
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summer bridge programs (i.e., EPCC, St. Philip’s College, South Texas College, and 
TAMIU), possibly due to differences in student recruitment strategies at the colleges. 



Texas Developmental Summer Bridge Programs 
Table 4.1 

Percentage of Study Participants Enrolled in 
One or More College Courses During the Summer 2009 Term 



Institution 


Program Group 
(%) 


Control Group 
(%) 


N 


El Paso Community College 


2.4 


9.3 


273 


Lone Star College-CyFair 


90.5 


33.3 


122 


Lone Star College-Kingwood 


90.2 


37.1 


86 


South Texas College 


16.9 


24.1 


137 


Texas A&M International University 


0.8 


3.5 


211 


Palo Alto College 


92.3 


8.6 


87 


San Antonio College 


59.6 


36.2 


147 


St. Philip’s College 


92.2 


4.9 


255 


Total 


47.2 


16.0 


1,318 



The Program Group Student Experience 

Data from the student baseline and post-program surveys, as well as comments 
made by students who participated in focus groups, illuminate the experience of students in 
the program group. The subsequent sections look at students’ motivation and goals, 
experiences with the program, engagement, and overall satisfaction. 

Student Motivation and Goais 

Student responses on the program intake form and in on-site focus group interviews 
conducted during the summer bridge programs indicate that students generally shared 
similar goals and motivations in seeking to participate in these programs. Most commonly, 
students who enrolled in the developmental summer bridge programs aimed to (1) prepare 
for college, (2) improve their academic performance, and (3) improve their performance on 
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standardized tests. Many students also suggested that they felt the developmental summer 
bridge programs would facilitate achievement of their long-term education and career goals. 

Students believed that participating in a developmental summer bridge program 
would help them prepare for college in a number of ways, particularly by helping them to 
become college-ready more quickly and allowing them to avoid taking remedial classes 
during their first year of college. Students commented during focus group sessions: “I didn’t 
want to take any remedial classes,” and “[my primary goal in the program was to] just get a 
higher grade and . . . not to take the remedial classes.” Additionally, students indicated that 
they were motivated by the desire to “get a feel for what college is like” and “gain 
experience about college.” 

Students also aspired to improve their academic and standardized test performance 
through participation in a developmental summer bridge program. While various academic 
subjects — primarily math, reading, and writing — were mentioned by students in focus 
group sessions as target areas for improvement, developing better skills in math seemed to 
be of particular concern for many. As one student explained, “I needed the help in math 
really bad. I really knew I need it [sic], so much more than it was my choice, it was almost a 
necessity.” Students generally saw participation in a developmental summer bridge program 
either as an opportunity to learn and understand classroom material with which they 
struggled previously, or as a “refresher.” One student described participation as a way to 
“catch little things that I missed here and there, take my time more.” 

Notably, some students in focus group sessions expressed that the $400 stipend 
offered for completing a developmental summer bridge program did not play a major role in 
their decision to pursue program participation. For these students, the opportunity to take 
remedial courses at low or no cost served as sufficient motivation for program enrollment. 
The financial incentive did, however, encourage many students to attend all required 
classes. According to one student, the financial stipend “motivates you kind of, like, to go 
to first period because they tell you if you don’t go to all your classes, you’re not going to 
get it. So it kind of makes you. Because if it was up to me, I wouldn’t go to specific 
classes.” 



Accelerated Format of Program 

Many students identified the accelerated time frame in which the developmental 
summer bridge programs took place as a central aspect of their experience. Students 
mentioned during focus group interviews that they felt the rigor of the classes stemmed not 
simply from the content but also from the increased workload and expectations of the 
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accelerated summer programs. In a focus group interview, one student stated, “Missing one 
day here would be like missing a whole week in high school.” 

Similar sentiments were voiced by several other program group students. While 
many liked the accelerated pace of the classes, some noted that the workload was greater 
than in high school. One student stated, “I like the fast pace because you won’t forget 
something from one class to the next,” while another noted, “It’s a lot more work. ... It’s a 
lot more serious. You only have five weeks and you only have three tests.” Students were 
especially cognizant of how the fast pace affected instmctional practices. Some commented 
that teachers sometimes seemed rushed or flustered in their attempts to get through the 
course material. 

College Knowledge/Preparation for College 

Many students appreciated how the programs prepared them to successfully 
navigate the college landscape. When asked in one focus group if they were feeling better 
prepared for college, student responses were overwhelmingly affirmative. One student 
stated, “Our student development teacher brought us to the library and showed us how to 
use resources.” In terms of college knowledge, students also mentioned learning how to 
register for courses, where to go for advising services, how to set appropriate goals, how to 
obtain help with improving basic skills, and how to take advantage of tutoring services. 
Although not all participants were enthusiastic about the college knowledge workshops and 
presentations, many students remarked that they felt better prepared to utilize campus 
resources and to navigate the college bureaucracies after program completion. 

Student-Faculty Relationships 

The opportunity to develop closer personal relationships with instructors was 
another factor that shaped program student experiences. As one student reported during a 
focus group interview: 

I’m pretty close with them [the instructors]. ... We have her number in case 
we can’t make it or something, or get sick, so it’s pretty close. It’s not like 
total strangers. 

Another student commented, “It’s a lot more personal. We feel comfortable with the 
instructor.” Other responses corroborated this sense of connection. According to one 
student, “The teachers can actually go around and help students. They look at what you’ve 
written down; they help you.” 

While students in many programs felt that they had an open and comfortable 
relationship with the faculty, other students mentioned that relationships with their teachers 
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were not as warm. One student pointed out that “half of them don’t even know our names,” 
while another student blamed the accelerated nature of the program for the more impersonal 
student-teacher relationships: “I think if the program was longer, like you get a closer bond 
with your teachers.” 

Student Learning 

Focus group interview responses conveyed how instructors worked to help students 
make connections between what they were learning in their courses and ways of applying 
what they learned. In considering the ways that these connections made the learning process 
more enjoyable, one student explained: 

And then I go to my student development class. ... I think that class I love 
the best because, like, I can relate to what he’s saying and that’s something 
that kind of like interests me [. . .] like today we talked about priorities and 
values. 

When asked whether any of their learning exercises related to real-world situations, 
one student stated, “Yeah, in reading a lot. She’s [instructor] getting us prepared for college 
... with the writing.” Students generally reported more widespread relevant and active 
learning in the reading and writing courses. In many of these courses, students had the 
opportunity to select their own topics for journal writing, and reading materials spanned a 
variety of subject matters. Some math instructors also attempted to use relevant course 
material. In response to a question about whether real-world examples were used in math 
class, one student responded, “My teacher — he does because he uses money. Because he’s 
like, he tells us, ‘People think math is hard, but once they relate it to money, they get it.’” 

Students across the sites also reported receiving more individualized attention from 
instructors in their developmental summer bridge program courses than in their high school 
experiences. One student commented, “It’s more individual ... and he helps you with what 
you don’t know. So it’s more like just for you.” Another student remark accentuated how 
some teachers seemed to relate to developmental summer bridge students more: “Because 
the teacher like — he can relate to us a lot more than like other teachers that just go and just 
teach you about the skills or whatever.” 

Student Overall Program Success and Completion 

Student focus group interview data and results from the student exit survey 
administered at each of the participating colleges indicate that many students perceived the 
developmental summer bridge programs and their experiences with them as successful. Of a 
total of 689 students who showed up for a developmental summer bridge program in the 
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first week of program implementation, 638 students (93 percent) persisted until the final 
week, completing the entire summer program (see Table 4.2). Additionally, when asked if 
they were planning on attending the college where they attended a developmental summer 
bridge, students in focus groups generally answered in the affirmative. 

Students generally agreed that they felt academically prepared for college, and 
almost all students felt that the developmental summer bridge programs helped improve 
their study skills and allowed them to learn more about academic support services (tutoring, 
math and writing labs, etc.) and financial aid options. In addition, many students felt more 
confident about doing college-level work and attending college after participating in a 
developmental summer bridge program. 

While student perceptions of the overall program experience were overwhelmingly 
positive, students also suggested there were avenues for further program improvement, in 
terms of both structure and delivery of services. For example, some students commented 
that they felt excessively “babied” at times. In one focus group interview, when asked how 
developmental summer bridge program classes compared with those in high school, 
students agreed that the amount of instruction and “hand holding” was similar to that the 
amount experienced in high school: 

Actually, it feels like Fm still in high school ... because [the instructor] 
holds our hand. . . . And he explains everything in step, by step, by step, by 
step and Fm like, “Oh, my goodness. I already — we got it. We got it 
yesterday when you explained it to us.” I got it in high school. I got it in 
junior high when I heard this. We got it. 

Some students also suggested that they might have benefited more from had 
program models encouraged stronger cohort relationships. Although many classes did 
provide structured opportunities for students to interact with their peers, little informal 
interaction was encouraged outside of the classroom, resulting in some students lacking a 
peer support network. 

Students generally indicated dislike for the programs’ early-morning start times and 
the number of consecutive hours of mental focus that the developmental summer bridge 
programs demanded. They also expressed dissatisfaction with the amount of homework in 
classes, commenting on the time commitment demanded by the programs, even outside of 
time spent in the classroom or lab. However, many students also appreciated the accelerated 
program format and the opportunities provided by an intensive academic summer program. 
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Texas Developmental Summer Bridge Programs 



Table 4,2 

Program Enrollment and Completion 



Institution 


Students in 
Program Group 


Ever Attended 
Program 


Completed 

Program 


Completion 
Rate (%) 


El Paso Community College 


165 


139 


138 


99 


Lone Star College-CyFair 


75 


65 


64 


98 


Lone Star College-Kingwood 


52 


51 


41 


80 


South Texas College 


83 


72 


63 


88 


Texas A&M International University 


126 


113 


111 


98 


Palo Alto College 


53 


52 


35 


67 


San Antonio College 


91 


58 


47 


81 


St. Philip’s College 


154 


146 


139 


95 


Total 


799 


689 


638 


93 
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Chapter 5 



Program Impacts on Educational Outcomes 



A key goal of the developmental summer bridge study is to test whether the 
developmental summer bridge programs — with their four main components — help 
students matriculate into college and perform better academically once enrolled. This 
chapter focuses on the impact of the developmental summer bridge programs on enrollment 
and academic progress for two semesters after students first enrolled in the study. This 
chapter describes the data sources used in the analysis of academic outcomes, provides an 
overview of these outcomes, and reviews findings on the programs’ impacts on academic 
outcomes. 



Key Impact Findings and Selected Outcomes 

The study yielded two key findings: 

• Program group students did not enroll in either the fall or spring 
semester at significantly different rates than control group students; 
enrollment rates were high for both groups. 

• There is evidence that the program group students were more likely to 
pass college-level courses in math and writing in the fall semester 
following the summer programs and more likely to attempt higher level 
reading, writing, and math courses than their control group counterparts. 

As described more fully in Box 5.1, the tables that follow present average outcomes for the 
students assigned to the program group and the control group, the difference between the 
two groups’ averages (which represents the estimated impact of the program), and the 
standard error of the difference. 

The indicators of student academic progress examined in this report reflect 
measures that are commonly viewed as important in the community college setting. The 
primary data source for each measure is the THECB. Data from the eight participating 
colleges are used to supplement the statewide THECB data unless otherwise noted. In order 
to reduce the likelihood of observing spurious relationships, we limited the number of 
primary (or confirmatory) outcomes examined (Schochet, 2008). The analyses of primary 
outcomes serve as the basis for determining the overall effectiveness of the developmental 
summer bridge programs, while the analyses of secondary outcomes are more exploratory 
in nature and serve to identify areas for future study. 
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Box 5.1 



How to Read the Impact Tables in This Report 

Most tables in this report use a similar format, illustrated below. The abbreviated table below 
displays transcript data and shows some educational outcomes for the program group and the control 
group. The first row, for example, shows that 15.1 percent of the program group members and 6.5 
percent of the control group members enrolled in a first college-level math course. 

Because individuals were assigned randomly either to the program group or to the control group, the 
effects of the program can be estimated by the difference in outcomes between the two groups. The 
“Difference” column in the table shows the differences between the two research groups’ outcomes 
— that is, the program’s estimated impacts on the outcomes. For example, the estimated impact on 
attempting the first college-level math course can be calculated by subtracting 6.5 percent from 15.1 
percent, yielding an increase or estimated impact of 8.7 percentage points (rounded). Thus the term 
impact refers to the “added value” of the program, or the program’s effects that go above and beyond 
the effects of the services provided to the control group. This difference represents the estimated 
impact rather than the true impact because, although study participants are randomly assigned to the 
program and control groups, there is still a possibility that differences could be observed by chance. 

Differences marked with one or more asterisks are statistically significant, meaning that there is only 
a small probability that the observed difference occurred by chance. The number of asterisks 
indicates the probability of observing differences at least as extreme as the observed differences if 
the program’s true impact is zero. One asterisk corresponds to a 10 percent probability; two asterisks, 
a 5 percent probability; and three asterisks, a 1 percent probability. For example, as the first row of 
the table excerpt shows, the program’s estimated impact on students enrolling in the first college- 
level math course is 8.7 percentage points. The three asterisks indicate that this difference is 
statistically significant at the 1 percent level, meaning that there is less than a 1 percent chance of 
observing a difference this large if the program’s true impact is zero. In other words, one can be 99 
percent confident that the program had a positive impact on students attempting the first college- 
level math course. 

The statistical significance is calculated using the standard error of the impact estimate, shown in the 
rightmost column. The standard error is a measure of uncertainty or variability around the impact 
estimate. Some useful rules of thumb are that there is about a 90 percent chance that the true impact 
is within plus or minus 1.65 standard errors of the estimated impact, roughly a 95 percent chance that 
the true impact is within plus or minus 1.96 standard errors of the estimated impact, and about a 99 
percent chance that the hue impact is within plus or minus 2.58 standard errors of the estimated 
impact. For example, in the first row of data below, there is roughly a 99 percent chance that the 
program’s impact on students’ likelihood of attempting the first college-level math course lies 
between 4.31 and 13.09 percentage points, calculated as 8.7 ± (2.58 x 1.7). 



Program Control Difference Standard 

Group (%) Group (%) (Impact) Error 

15.1 6.5 8 . 7 *** 1.7 

8.8 4.4 4 . 4 *** 14 



Outcome 

Attempted first college-level math course 
Passed first college-level math course 
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Primary Indicators of Academic Progress 

• Fall 2009 enrollment: One of the goals of the developmental summer 
bridge programs was to ease the transition into college for students. 

Thus, an important indicator of the success of the program is enrollment 
at any college in the fall 2009 term. 

• Progression in developmental and college-level courses in math, 
reading, and writing: One of the goals of the developmental summer 
bridge programs was to accelerate students’ progress through any 
required developmental course sequence and through the first college- 
level course in math, reading, or writing. An important indicator of this 
progress is the completion of entry-level college courses. 

Secondary/Exploratory Indicators of Academic Progress 

• Spring 2010 enrollment: This outcome measures the enrollment rate at 
any college in the spring 2010 semester. 

• Total credits attempted: Since a student’s progress toward earning a 
certificate or degree depends on fulfilling both developmental and 
college-level requirements, it is difficult to measure progress toward a 
degree directly. For this reason, the number of credits attempted will 
serve as a proxy for progress toward a degree. The primary measure of 
this progress will be the total number of credits attempted 
(developmental and college-level) in fall 2009, spring 2010, and 
cumulatively over the follow-up period, through the spring of 2011. 
Additionally, the number of developmental credits attempted and 
college-level credits attempted will be examined separately. 

• Developmental credits earned: An examination of the number of 
developmental credits accumulated in the fall 2009 term can be used to 
gain a better understanding of whether students who attempt credits 
indeed pass the course and earn the credits. While it would be ideal to 
present information on both developmental and college-level credits 
earned, the latter are not currently available. We expect to have 
measures of college-level credits earned in the final report. 

In addition, student progress measures are examined by subgroup to explore (1) whether the 
programs are equally effective for different types of students and (2) whether different types 
of summer bridge programs are equally effective. In the current report, exploratory 
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subgroup analyses are conducted to assess differential program effectiveness (1) for males 
and females and (2) for students in course-based and freestanding programs. The relative 
performance of student groups that differ by socioeconomic status will be analyzed in the 
final report. 



Fall 2009 Semester Outcomes 

Table 5.1 shows select academic outcomes from the fall 2009 semester — the first 
semester in which students were eligible to enroll in college after the summer bridge 
programs — and enrollment outcomes from the spring 2010 semester.^ 

The analyses described in this report are intent-to-treat (ITT) analyses, which 
examine the impact of being assigned to the program group rather than the impact of 
receiving the program.^ In other words, the program group’s results are for all 793 students 
who were assigned to the program group, including the 104 students who did not participate 
in a summer bridge program, in addition to the 689 students who did participate. 

Enrollment 

Surprisingly, programs did not have any impact on fall 2009 registration rates; 
students in the program group registered for courses in the fall 2009 semester at a rate (77.4 
percent) that is statistically indistinguishable from the enrollment rate of the control group 
(76.4 percent). This finding contradicts the hypothesis that the summer bridge programs 
would boost enrollment rates among the program group students. 

Progression in Math 

While students in the program and control groups attempted at least one math 
course at similar rates, students who participated in the developmental summer bridge 
programs went on to attempt the first college-level math course at a significantly^ higher 
rate (15.1 percent) than students in the control group (6.5 percent). A significantly higher 
percentage of program group students also passed this first college-level math course (8.8 
percent, compared with 4.4 percent for the control group), though this difference is due to 
the larger percentage of program group students who enrolled in an entry-level college math 
course. 



*See Appendix A for a detailed description of the analytic model used to estimate impacts. 
^For a detailed description of the difference between these two types of analyses 
interpretations, see Bloom (2005). 

^Throughout this report, the terms “significant” and “significantly” always refer to 
significance. See Box 5.1. 



and their 
statistical 
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Texas Developmental Summer Bridge Programs 



Table 5.1 



Select Academic Outcome Measures 



Outcome 


Program 

Group 


Control 

Group 


Difference 

(Impact) 


p- 

value 


Std. 

Error 


N 


793 


525 








Fall 2009 


Registered for at least one course 


77.4% 


76.4% 


1.0 


.68 


2.4 


Math progression 
Attempts 


First college-level course 


15.1% 


6.5% 




.00 


1.7 


1 level below college-level 


14.6% 


16.9% 


-2.3 


.24 


2.0 


2 levels below college-level 


9.9% 


11.5% 


-1.6 


.35 


1.7 


3 levels below college-level 


8.0% 


10.5% 


-2.5 


.12 


1.6 


Did not attempt any level 


52.3% 


54.6% 


-2.3 


.38 


2.7 


Passes 


First college-level course 


8.8% 


4.4% 




.00 


1.4 


1 level below college-level 


10.5% 


12.2% 


-1.7 


.32 


1.7 


2 levels below college-level 


7.3% 


8.2% 


-1.0 


.52 


1.5 


3 levels below college-level 


6.9% 


8.0% 


-1.1 


.45 


1.4 


Did not pass any attempts 


14.3% 


12.6% 


1.7 


.36 


1.9 


Reading progression 
Attempts 


First college-level course 


42.0% 


36.2% 


5 


.02 


2.5 


1 level below college-level 


10.0% 


9.4% 


0.6 


.72 


1.7 


2 levels below college-level 


4.2% 


4.8% 


-0.6 


.60 


1.2 


3 levels below college-level 


1.9% 


4.4% 


_2 5*** 


.01 


0.9 


Did not attempt any level 


42.0% 


45.3% 


-3.2 


.22 


2.6 


Passes 


First college-level course 


30.6% 


27.1% 


3.5 


.15 


2.4 


1 level below college-level 


7.9% 


7.8% 


0.1 


.95 


1.5 


2 levels below college-level 


3.7% 


4.6% 


-0.9 


.40 


1.1 


3 levels below college-level 


1.4% 


3.8% 


-2 4*** 


.00 


0.8 


Did not pass any attempts 


14.4% 


11.4% 


3.0 


.11 


1.9 
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Table 5.1 (continued) 



Outcome 


Program 

Group 


Control 

Group 


Difference 

(Impact) 


P- 

value 


Std. 

Error 


Writing progression 
Attempts 

First college-level course 


39.7% 


35.8% 


3.9 


.13 


2.6 


1 level below college-level 


12.5% 


8.8% 


2 y** 


.03 


1.7 


2 levels below college-level 


2.1% 


1.2% 


1.0 


.18 


0.7 


3 levels below college-level 


1.4% 


2.1% 


-0.7 


.32 


0.7 


Did not attempt any level 


44.3% 


52.1% 




.00 


2.6 


Passes 

First college-level course 


32.0% 


26.9% 




.04 


2.5 


1 level below college-level 


8.7% 


7.2% 


1.5 


.33 


1.5 


2 levels below college-level 


1.9% 


0.8% 


1.1* 


.09 


0.7 


3 levels below college-level 


1.1% 


2.1% 


-1.0 


.16 


0.7 


Did not pass any attempts 


12.0% 


10.9% 


1.1 


.54 


1.8 


Credits attempted 


9.2 


8.8 


0.3 


.29 


0.3 


College credits 


6.2 


5.9 


0.3 


21 


0.3 


Developmental credits 


3.0 


3.0 


0.0 


.89 


0.2 


Developmental credits earned 


1.8 


1.9 


-0.1 


.66 


0.1 


Snrina 2010 


Registered for any courses 


74.5% 


75.3% 


-0.8 


.75 


2.4 


Credits attempted 


9.3 


9.1 


0.3 


.49 


0.4 


College credits 


7.7 


7.1 


* 

80 

O 


.09 


0.3 


Developmental credits 


1.6 


2.0 


-0.3** 


.03 


0.1 


Cumulative Measures^ 


Credits attempted 


19.9 


18.7 


1.1* 


.07 


0.6 


College credits 


14.3 


13.3 


1.0* 


.07 


0.6 


Developmental credits 


5.5 


5.4 


0.1 


.69 


0.3 



SOURCE: NCPR calculations using Texas Higher Education Coordinating Board data and college transcript 
data. 



NOTES: Rounding may cause slight discrepancies in sums and differences. Estimates are adjusted by site. A 
two-tailed Mest was applied to differences between research groups. 

***/><.01. **p<.05. *p<A0. 

“Includes credits attempted in summer 2009. 
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Progression in Reading 

There is no statistically significant difference in the rate at which students in the 
program and control groups attempted at least one reading course. However, program group 
students were significantly more likely to attempt a college-level reading course (the rate 
was 42.0 percent, compared with 36.2 percent for the control group) and significantly less 
likely to attempt the lowest level course in developmental reading (the rate was 1.9 percent, 
compared with 4.4 percent for the control group). It is worth noting, however, that the 
difference in pass rates for the first college-level reading course among program group 
students (30.6 percent) and control group students (27.1 percent) is not statistically 
significant. 

Progression in Writing 

Significantly more program group students than control group students attempted at 
least one writing course (the rates were 55.7 percent and 47.9 percent, respectively). 
Program group students were also more likely to attempt the highest level of developmental 
writing compared with the students in the control group (the rates were 12.5 percent and 8.8 
percent, respectively). Students in the program group were significantly more likely to pass 
both their first college-level writing course (the rate was 32.0 percent, compared with 26.9 
percent for the control group) and a developmental writing course two levels below the 
college level (the rate was 1.9 percent, compared with 0.8 percent for the control group; the 
1.1 percentage point difference here is statistically significant at the 10 percent level). 

Total Credits Attempted 

There is no statistically significant difference between the two groups in the total 
number of credits attempted in fall 2009 or in the composition of their respective credit 
loads in fall 2009. Students in the program group registered for about the same number of 
college-level credits (about 6 credits) and developmental credits (about 3 credits) as 
students in the control group. 

Developmental Credits Earned 

There is no statistically significant difference between the two groups in the total 
number of developmental credits earned. Students in the program group earned, on average, 
1.8 developmental credits; students in the control group earned 1.9 developmental credits. 
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Spring 2010 Semester Outcomes and Cumulative Measures 



Enrollment 

Students in the program and control groups registered at similar rates in the spring 
2010 semester (the second full academic term following the developmental summer bridge 
programs). Surprisingly, registration rates for the spring 2010 semester (about 75 percent 
for both groups) were similar to those for the fall 2009 semester. Attrition is common 
between the fall and spring semesters; the lack of attrition seen here is attributable to the 
fact that some students who did not register for the fall 2009 term registered for the spring 
2010 term, while other students who did register in fall 2009 did not return in the spring. 

Total Credits Attempted 

There is a statistically significant difference in the number of college-level and 
developmental credits attempted in the spring semester by students in the program group 
compared with students in the control group. Students in the program group registered for 
more college-level credits (7.7 credits, compared with 7.1 credits for the control group; this 
0.6 credit impact is statistically significant at the 10 percent level) and fewer developmental 
credits (1.6 credits, compared with 2.0 credits for the control group) than control group 
students. This may reflect the program’s effectiveness in getting students to attempt higher 
level courses in math, reading, or writing sequences. 

Cumulative Measures 

As of the spring 2010 semester, students in the program group had attempted, on 
average, one more college-level credit than did students in the control group (program 
group students attempted 14.3 credits, compared with 13.3 credits for the control group; the 
1.0 credit impact is statistically significant at the 10 percent level). Follow-up analyses will 
compare the two groups in terms of college-level and developmental credits attempted and 
earned in the two years following the summer bridge program. 



Subgroup Analyses 

This section presents impacts on educational outcomes by subgroup (seen also in 
Tables 5.2 and 5.3). These analyses assess whether there were impacts for the selected 
subgroups of students (e.g., females) and whether there were differential impacts between 
subgroups of students (e.g., between females and males). Subgroup analyses were 
conducted to ascertain differences in the program’s effect on males and females because 
differences have been detected in other developmental education interventions (Scrivener et 
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al., 2008). A subgroup analysis was also performed to compare the impacts of freestanding 
and course-based programs. 

Academic Outcomes by Gender 

There are almost no statistically significant differences in impacts between men and 
women on the primary outcome measures of the effectiveness of the developmental summer 
bridge programs (the sole exception is in passing the lowest level of developmental 
reading), and there are no statistically significant differences in impacts for secondary 
measures of the programs’ effectiveness. In other words, there is virtually no evidence that 
the programs were more or less effective for either gender. 

Academic Outcomes by Program Type 

There is some evidence that course-based and freestanding developmental summer 
bridge programs had different impacts on program and control group students. For example, 
the two types of programs had significantly different impacts on the number of 
developmental credits students attempted in the fall 2009 semester, with a positive impact 
(of 0.6 credits) for students attending the course-based programs and no statistically 
significant impact (the statistically non-significant estimate is -0.3 credits) for those 
attending the freestanding programs. There are also statistically significant differences 
between the impacts of course-based and freestanding programs on the number of credits 
students attempted in the spring 2010 term and on the types of courses they registered for. 
For example, whereas students in the course-based programs attempted, on average, 1.8 
more college-level credits than students in the control group, there is no statistically 
significant difference in the number of college-level credits attempted by students in the 
freestanding programs compared with students in the control group. The results suggest that 
the course-based programs may have been more effective in getting students to attempt 
more college-level courses. While it is not surprising that participation in a course-based 
program results in a higher number of credits attempted, program group students in the 
course-based programs continued to register for more credits than their control group 
counterparts in the post-program semesters. 
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Table 5.2 



Academic Outcomes by Gender 



Outcome (with significant differences 
between subgroups denoted by daggers) 






Male 








Female 








Program 

Group 


Control 

Group 


Diff 

(Impact) 


P- 

value 


Std. 

Error 


Program 

Group 


Control 

Group 


Diff 

(Impact) 


P- 

value 


Std. 

Error 


N 


293 


196 








489 


321 








Fall 2009 


Registered for at least one course 


79.1% 


79.7% 


-0.5 


.88 


3.7 


76.3% 


74.2% 


2.1 


.49 


3.1 


Math progression 
Attempts 


First college-level course 


15.8% 


8.0% 




.01 


2.9 


14.4% 


5.4% 




.00 


2.1 


1 level below college-level 


15.5% 


18.7% 


-3.3 


.32 


3.3 


14.1% 


15.9% 


-1.8 


.47 


2.4 


2 levels below college-level 


8.5% 


8.3% 


0.2 


.93 


2.5 


10.7% 


13.6% 


-2.8 


.22 


2.3 


3 levels below college-level 


8.6% 


9.0% 


-0.4 


.88 


2.6 


7.7% 


11.6% 


-3.9* 


.06 


2.1 


Did not attempt any level 


51.6% 


56.0% 


-4.4 


.32 


4.4 


53.0% 


53.5% 


-0.5 


.88 


3.4 


Passes 


First college-level course 


9.1% 


4.7% 


4.4* 


.06 


2.4 


8.9% 


3.9% 


5 QJissisJis 


.01 


1.8 


1 level below college-level 


9.9% 


14.8% 


-4.9* 


.10 


2.9 


10.5% 


11.1% 


-0.6 


.79 


2.1 


2 levels below college-level 


5.5% 


5.1% 


0.4 


.83 


2.1 


8.3% 


10.2% 


-1.9 


.35 


2.0 


3 levels below college-level 


7.3% 


6.0% 


1.3 


.57 


2.3 


6.7% 


9.4% 


-2.7 


.15 


1.9 


Did not pass any attempts 


16.6% 


13.5% 


3.1 


.33 


3.2 


12.7% 


11.9% 


0.8 


.73 


2.3 




Table 5.2 (continued) 



Outcome (with significant differences 
between subgroups denoted by daggers) 






Male 








Female 








Program 

Group 


Control 

Group 


Diff 

(Impact) 


P- 

value 


Std. 

Error 


Program 

Group 


Control 

Group 


Diff 

(Impact) 


P- 

value 


Std. 

Error 


Reading progression 
Attempts 


First college-level course 


42.0% 


36.7% 


5.4 


.18 


4.0 


41.6% 


36.0% 


5.6* 


.09 


3.3 


1 level below college-level 


9.8% 


6.2% 


3.6 


.16 


2.6 


10.3% 


11.1% 


-0.9 


.69 


2.2 


2 levels below college-level 


3.4% 


5.6% 


-2.2 


.24 


1.9 


4.7% 


4.4% 


0.3 


.85 


1.5 


3 levels below college-level 


1.4% 


2.1% 


-0.7 


.56 


1.2 


2.0% 


5.9% 




.00 


1.3 


Did not attempt any level 


43.4% 


49.5% 


-6.1 


.14 


4.1 


41.4% 


42.5% 


-1.1 


.75 


3.4 


Passes 


First college-level course 


29.2% 


27.2% 


2.0 


.61 


3.8 


31.5% 


27.4% 


4.1 


.18 


3.1 


1 level below college-level 


6.5% 


5.6% 


0.8 


.71 


2.2 


9.0% 


9.0% 


0.0 


.99 


2.1 


2 levels below college-level 


3.1% 


5.1% 


-2.0 


.25 


1.8 


4.1% 


4.4% 


-0.4 


.80 


1.4 


3 levels below college-level ft 


1.4% 


1.5% 


-0.1 


.91 


1.1 


1.2% 


5.3% 


* 


.00 


1.2 


Did not pass any attempts 


16.5% 


11.0% 


5.5* 


.08 


3.1 


12.8% 


11.4% 


1.4 


.55 


2.3 


Writing progression 
Attempts 


First college-level course 


38.5% 


36.9% 


1.6 


.71 


4.2 


40.3% 


35.1% 


5.2 


.12 


3.3 


1 level below college-level 


14.0% 


8.6% 


5.4* 


.07 


2.9 


11.2% 


9.1% 


2.0 


.35 


2.2 


2 levels below college-level 


3.1% 


0.5% 


2.6* 


.05 


1.3 


1.7% 


1.5% 


0.1 


.87 


0.9 


3 levels below college-level 


1.0% 


2.0% 


-1.0 


.36 


1.1 


1.6% 


1.9% 


-0.3 


.78 


0.9 


Did not attempt any level 


43.4% 


51.9% 


-8.5** 


.04 


4.2 


45.2% 


52.3% 


_7 


.03 


3.3 


Passes 


First college-level course 


30.4% 


27.0% 


3.4 


.40 


4.0 


33.4% 


27.1% 


6.3** 


.05 


3.2 


1 level below college-level 


8.3% 


5.5% 


2.8 


.25 


2.4 


8.5% 


8.5% 


0.0 


.99 


2.0 


2 levels below college-level 


2.4% 


0.5% 


1.9 


.11 


1.2 


1.7% 


0.9% 


0.7 


.37 


0.8 




Table 5.2 (continued) 



Outcome (with significant differences 
between subgroups denoted by daggers) 






Male 








Female 








Program 

Group 


Control 

Group 


Diff 

(Impact) 


P- 

value 


Std. 

Error 


Program 

Group 


Control 

Group 


Diff. 

(Impact) 


P- 

value 


Std. 

Error 


3 levels below college-level 


1.0% 


2.0% 


-1.0 


.36 


1.1 


1.2% 


1.9% 


-0.7 


.43 


0.9 


Did not pass any attempts 


14.5% 


13.0% 


1.5 


.63 


3.1 


10.0% 


9.3% 


0.7 


.74 


2.1 


Credits attempted 


9.3 


9.3 


0.1 


.87 


0.5 


9.1 


8.5 


0.5 


.19 


0.4 


College credits 


6.4 


6.4 


0.0 


.95 


0.4 


6.0 


5.5 


0.5 


.14 


0.3 


Developmental credits 


2.9 


2.9 


0.0 


.87 


0.3 


3.1 


3.1 


0.0 


.96 


0.2 


Developmental credits earned 


1.6 


1.6 


0.0 


.98 


0.2 


1.9 


2.1 


-0.1 


.45 


0.2 


Spring 2010 


Registered for at least one course 


77.3% 


79.3% 


-2.0 


.59 


3.8 


73.0% 


72.6% 


0.4 


.90 


3.2 


Credits attempted 


9.4 


9.6 


-0.2 


.72 


0.6 


9.4 


8.8 


0.6 


.21 


0.5 


College credits 


7.9 


7.7 


0.2 


.70 


0.5 


7.6 


6.8 


0.8* 


.05 


0.4 


Developmental credits 


1.5 


1.9 


-0.4 


.10 


0.2 


1.7 


2.0 


-0.3 


.18 


0.2 


Cumulative Measures^ 


Credits attempted 


20.1 


19.4 


0.6 


.52 


1.0 


19.8 


18.2 


1.6* 


.05 


0.8 


College credits 


14.8 


14.3 


0.4 


.63 


0.9 


14.1 


12.6 




.04 


0.7 


Developmental credits 


5.3 


5.1 


0.2 


.71 


0.5 


5.7 


5.6 


0.1 


.74 


0.4 



SOURCE: NCPR calculations from Texas Higher Education Coordinating Board and college transcript data. 



NOTES: Rounding may cause slight discrepancies in sums and differences. Estimates are adjusted by site. A two-tailed t-test was applied to differences 
between research groups and differences of impacts between subgroups. 

***p < .01. **p < .05. *p < .10. 

Significant differences between subgroups denoted as follows: < -01; -05; V -lO- 

“Includes credits attempted in summer 2009. 
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Table 5.3 



Academic Outcomes by Program Type 



Outcome (with significant differences 
between subgroups denoted by daggers) 




Freestanding Programs 








Course-Based Programs 








Program 

Group 


Control 

Group 


Diff 

(Impact) 


P- 

value 


Std. 

Error 


Program 

Group 


Control 

Group 


Diff 

(Impact) 


P- 

value 


Std. 

Error 


N 


527 


349 








266 


176 








Fall 2009 


Registered for at least one course 


76.8% 


75.7% 


1.1 


.70 


2.9 


78.6% 


77.9% 


0.7 


.86 


4.0 


Math progression 
Attempts 


First college-level course 


16.0% 


5.4% 


IQ 


.00 


2.1 


13.5% 


8.5% 


5.0* 


.10 


3.0 


1 level below college-level 


13.1% 


16.6% 


-3.5 


.13 


2.4 


17.7% 


17.5% 


0.2 


.96 


3.5 


2 levels below college-level 


10.7% 


13.3% 


-2.5 


.24 


2.2 


8.3% 


8.0% 


0.3 


.90 


2.6 


3 levels below college-level 


7.2% 


10.1% 


-2.9 


.13 


1.9 


9.8% 


11.4% 


-1.6 


.58 


2.9 


Did not attempt any level 


53.0% 


54.6% 


-1.6 


.64 


3.3 


50.7% 


54.6% 


-3.8 


.39 


4.5 


Passes 


First college-level course 


9.3% 


4.0% 


5 


.00 


1.7 


7.9% 


5.1% 


2.7 


.26 


2.4 


1 level below college-level 


10.0% 


12.1% 


-2.0 


.33 


2.1 


11.3% 


12.4% 


-1.1 


.71 


3.0 


2 levels below college-level 


7.0% 


8.7% 


-1.7 


.35 


1.8 


7.9% 


7.4% 


0.5 


.84 


2.5 


3 levels below college-level 


6.4% 


8.3% 


-1.9 


.28 


1.8 


7.9% 


7.4% 


0.5 


.85 


2.6 


Did not pass any attempts 


14.2% 


12.3% 


1.9 


.40 


2.3 


14.3% 


13.1% 


1.2 


.71 


3.3 




Table 5.3 (continued) 



Outcome (with significant differences 
between subgroups denoted by daggers) 




Freestanding Programs 








Course-Based Programs 








Program 

Group 


Control 

Group 


Diff 

(Impact) 


P- 

value 


Std. 

Error 


Program 

Group 


Control 

Group 


Diff 

(Impact) 


P- 

value 


Std. 

Error 


Reading progression 
Attempts 


First college-level course 


41.3% 


34.7% 


6.6** 


.03 


3.0 


43.2% 


39.2% 


4.0 


.38 


4.6 


1 level below college-level 


8.2% 


8.6% 


-0.4 


.82 


1.9 


13.5% 


10.8% 


2.7 


.40 


3.2 


2 levels below college-level ftt 


5.3% 


3.7% 


1.6 


.28 


1.4 


1.9% 


6.8% 




.01 


1.9 


3 levels below college-level ft 


1.9% 


5.7% 


-3 8*** 


.00 


1.2 


1.9% 


1.7% 


0.2 


.90 


1.3 


Did not attempt any level 


43.3% 


47.2% 


-3.9 


.23 


3.2 


39.5% 


41.4% 


-1.9 


.66 


4.5 


Passes 


First college-level course 


30.5% 


26.2% 


4.3 


.14 


2.9 


30.8% 


29.0% 


1.9 


.66 


4.3 


1 level below college-level f 


6.3% 


8.0% 


-1.8 


.31 


1.8 


11.3% 


7.4% 


3.8 


.18 


2.9 


2 levels below college-level ftt 


4.5% 


3.4% 


1.1 


.42 


1.4 


1.9% 


6.8% 




.01 


1.9 


3 levels below college-level ft 


1.3% 


4.9% 


-3.6*** 


.00 


1.1 


1.5% 


1.7% 


-0.2 


.87 


1.2 


Did not pass any attempts 


14.1% 


10.2% 


3.8* 


.09 


2.2 


15.0% 


13.7% 


1.4 


.69 


3.4 


Writing progression 
Attempts 


First college-level course 


39.1% 


34.4 


4.7 


.13 


3.1 


40.9% 


38.7% 


2.2 


.63 


4.6 


1 level below college-level 


9.5% 


7.5 


2.0 


.30 


1.9 


18.4% 


11.3% 


y 1 ** 


.04 


3.5 


2 levels below college-level 


2.1% 


0.6 


1.5* 


.07 


0.8 


2.2% 


2.3% 


0.0 


.98 


1.4 


3 levels below college-level 


2.1% 


2.6 


-0.5 


.63 


1.0 


0.0% 


1.1% 


-1.1* 


.08 


0.7 


Did not attempt any level 


47.3% 


55.0 




.02 


3.2 


38.4% 


46.5% 


-8.2* 


.07 


4.4 


Passes 


First college-level course 


31.7% 


25.8 


5 9 ** 


.05 


2.9 


32.7% 


29.0% 


3.7 


.41 


4.5 


1 level below college-level 


6.1% 


5.7 


0.4 


.82 


1.6 


13.9% 


10.2% 


3.7 


.25 


3.2 


2 levels below college-level 


1.7% 


0.3 


1.4* 


.05 


0.7 


2.2% 


1.7% 


0.5 


.69 


1.4 


3 levels below college-level 


1.7% 


2.6 


-0.9 


.37 


1.0 


0.0% 


1.1% 


-1.1* 


.08 


0.7 


Did not pass any attempts 


11.6% 


10.6 


0.9 


.66 


2.1 


12.8% 


11.4% 


1.4 


.66 


3.1 




Table 5.3 (continued) 



Outcome (with significant differences 
between subgroups denoted by daggers) 




Freestanding Programs 








Course-Based Programs 








Program 

Group 


Control 

Group 


Diff 

(Impact) 


P- 

value 


Std. 

Error 


Program 

Group 


Control 

Group 


Diff 

(Impact) 


p- 

value 


Std. 

Error 


Credits attempted 


8.9 


8.8 


0.1 


.87 


0.4 


9.7 


8.9 


0.8 


.11 


0.5 


College credits 


6.2 


5.8 


0.4 


.29 


0.3 


6.1 


5.9 


0.2 


.68 


0.5 


Developmental credits ft 


2.7 


3.0 


-0.3 


.20 


0.2 


3.6 


3.0 


0.6** 


.03 


0.3 


Developmental credits earned ft 


1.6 


1.8 


-0.3 


.11 


0.2 


2.4 


2.0 


0.3 


.18 


0.3 


Soring 2010 


Registered for at least one course 


73.4 


74.8 


-1.4 


.64 


3.0 


76.7% 


76.2% 


0.5% 


.90 


4.1 


Credits attempted ftt 


8.5 


9.1 


-0.6 


.18 


0.4 


11.0 


9.1 




.01 


0.7 


College credits ft 


7.1 


7.1 


0.0 


.92 


0.4 


9.0 


7.2 




.01 


0.7 


Developmental credits f 


1.5 


2.0 


_Q 


.00 


0.2 


2.0 


2.0 


0.0 


.86 


0.3 


Cumulative Measures^ 


Credits attempted ftt 


17.9 


18.4 


-0.5 


.50 


0.7 


23.7 


19.4 


4 2 *** 


.00 


1.1 


College credits 


13.7 


13.2 


0.5 


.48 


0.7 


15.6 


13.5 


2.1** 


.05 


1.0 


Developmental credits ftt 


4.2 


5.2 




.00 


0.3 


8.1 


5.9 


2 2*** 


.00 


0.5 



SOURCE: NCPR calculations using Texas Higher Education Coordinating Board and college transcript data. 



NOTES: Rounding may cause slight discrepancies in sums and differences. Estimates are adjusted by site. A two-tailed t-test was applied to differences between 
research groups and differences of impacts between subgroups. 

***p < .01. **p < .05. *p < .10. 

Significant differences between subgroups denoted as follows: < -01; < -05; V < -lO- 

includes credits attempted in summer 2009. 




Chapter 6 



Conclusion: A Positive Early Look 



Our conclusion from this early look is that the Texas developmental summer bridge 
programs we investigated did help students attempt and pass more college-level courses in 
math and writing. Specifically, by the end of spring 2010, students in the program group 
attempted one college-level credit more than students in the control group (statistically 
significant at the 10 percent level). This modest effect is encouraging, as students’ course- 
taking patterns appear to be trending in the desired direction — toward the accumulation of 
more college-level credits. It is important to note that these early findings reflect student 
academic progress for only one year and that longer follow-up will provide additional 
evidence. 

It is also worth noting that programs had no effect on enrollment rates for the fall or 
the spring, suggesting that they were not useful in enhancing access to college. Interpreted 
differently, the relatively high enrollment rates for both program group and control group 
students might reflect the behaviors of a highly motivated student population, already on 
track to go to college. In fact, students stated on the intake form (and program students 
reiterated in their focus groups) a desire to attend and succeed in college. Based on the 
evidence from this study, it is not possible to determine whether students who were less 
likely to attend college would have benefited from the programs. 



The Developmental Summer Bridge Model 

The research in this report suggests that, although the delivery of the summer bridge 
program varied somewhat across colleges, each of the eight colleges implemented a fair test 
of the program model. Because each of the four components of the program model are 
integral to the intervention, teasing apart which programmatic elements were most 
successful in promoting positive student outcomes is not feasible in the current research. 
However, there is modest support for the notion that course-based programs are slightly 
better than freestanding programs at helping students attempt college-level courses. This 
may reflect the fact that more students in the course-based programs entered college at 
higher levels of the developmental course sequence. With the exception of Palo Alto, 
course-based programs primarily targeted students at the highest developmental levels, 
while freestanding programs invited a broader range of developmental students to 
participate. Another interpretation of these findings is that because the course-based 
programs actually award developmental credit, students in these programs may be 
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advantaged in the number of credits attempted. Nonetheless, these findings are preliminary, 
and we will revisit these program impacts with longer follow-up in the final report. 



Implications for Policy and Practice 

Early findings imply that developmental summer bridge programs help students 
pass entry-level college courses in math and writing. But why are there not similar effects 
on students’ achievements in reading courses? It is possible that math and writing skills 
may be easier to remediate in college. If these results persist in the final year of the study, 
they may suggest that colleges should focus on building students’ skills in math and writing 
— the two subjects areas where remediation is often needed the most. These early results 
also suggest that Texas should consider continued funding for these programs — perhaps 
with incentive funding to encourage greater student participation and program completion. 
The question remains as to whether a stipend is necessary to recruit eligible students, since 
some students avail themselves of the programs without monetary incentives. 



Final Follow-Up Report 

A final report with two years of longitudinal follow-up will be released within the 
next year. We expect to learn more about students’ progression through developmental 
education, their success in college-level courses, and their persistence into and through the 
second year of college. 
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Appendix A 

Estimating Effects 




This appendix provides a detailed discussion of the statistical model used to 
estimate the program impacts on students’ postsecondary academic performance and other 
related statistical issues. 



Analysis of Program Impacts 

The program impact analysis involves examining outcome measures constructed 
from the college transcript data and data from the Texas Higher Education Coordinating 
Board (THECB), with key outcomes listed in Chapter 5. Note that all the listed outcomes 
are measured at the level of the individual student. The analyses use ordinary least squares 
(OLS) regression to estimate the average difference in outcomes between program and 
control group students. Group means are presented using SAS’s ESMEANS. The impact 
model includes college fixed effects; this means that the findings represent the estimated 
impact of the developmental summer bridge program in the set of colleges in which these 
interventions were implemented. In other words, the impact estimates are not generalizable 
to other colleges. 

The Model 

The impact analyses are pooled across colleges, with site-level dummies. For each 
outcome, the basic model used in the analysis is the following: 

( 1 ) 

K 



Where: 



Y. = Outcome of interest for student i 

Ti = Indicator of treatment group membership (treatment status). This 
indicator is equal to 1 if student i was assigned to the summer bridge 
program and zero otherwise 

= A dummy equal to 1 if student i is at campus k and zero otherwise 
= A within-student error term 

Therefore: 

= The estimated impact of the program on outcome Y 
There are several features to note about this model: 
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• Interpretation of the impact estimate. is a fixed-effect impact 
estimate used in order to consider the average impact of the 
developmental summer bridge program for students in the analysis 
sample. This approach was taken because this study most closely 
reflects an efficacy study of the effects of a summer bridge program 
under relatively controlled conditions.* 

• College indicators. Indicators for each college ( Ct ) are included in the 
model to capture a central feature of the research design, in which 
random assignment was conducted separately for each college. These 
indicators also account for variation in mean outcome levels across 
colleges. Thus, after including the college indicators in the model, the 
only source of variation in Y in this model is between students within 
colleges. 

Note that the model does not control for pre-random assignment achievement, as no such 
measure was available. 

Other Analytical Issues 

Handling Missing Outcome Data 

It is important to note that most outcomes of interest are influenced by enrollment 
rates. For example, only students who enroll have the opportunity to cam college credits, 
complete developmental course sequences, and utilize student services. Students who did 
not enroll are assigned a value of zero for all outcomes in all follow-up periods. Thus, low 
program group enrollment rates may reduce the likelihood of finding statistically significant 
impacts on other outcomes. 

Cases in which no enrollment data for a student is found are imputed a value of 
zero; that is, they are treated as though the student did not register. It is possible that a 
student who does not appear in the data enrolled at a college outside of Texas (and is thus 
absent from the THECB data). It is also possible that a student who is not found opted out 
of reporting at his or her college. However, in these analyses, absence of evidence is treated 
as evidence of absence, and we initially assumed the student did not enroll. A phone survey 
of students whose data were not available from the colleges or the THECB was conducted 

'Programs selected for the study were required to have four features: four to six weeks of accelerated 
instruction in developmental coursework; a college knowledge component; a stipend of up to $400; and 
additional student supports. Program features at other colleges may vary significantly, and the impact 
estimates described in this report are not statistically generalizable to a larger population of colleges or 
students. 



64 




to either confirm this assumption or fill in missing data about students’ college enrollment 
status. 



In cases where course information (such as the number of credits conferred) is missing 
from the files provided by colleges, we consulted college course catalogs to determine the 
appropriate values. Observations that are missing information on course completion (such as 
grades) were checked with the colleges. In cases where no grade could be determined, a grade 
of “incomplete” is assigned. 

Approaching Multiple Hypothesis Testing 

When making judgments about the statistical significance of the estimated impacts, it is 
important to recognize the potential problems associated with conducting multiple hypothesis 
tests. Conducting hypothesis tests for estimated impacts on several different outcomes and for 
many subgroups of students or schools increases the likelihood of concluding that some impact 
estimates are statistically significant when in fact they are not (generating a Type I error, or false 
positive). 

We use two sets of safeguards to attenuate the risk of drawing inappropriate conclusions 
based on false positives. First, we use a parsimonious set of outcome measures. Second, we 
identify a priori a set of primary outcomes and subgroups. The primary outcome measures and 
subgroups are presented in Chapter 5. 



65 




References 



Adelman, C. (1998). The kiss of death? An alternative view of college remediation. 
National Crosstalk, 6(3). Retrieved from http://www.highereducation.org/crosstalk/ 
ct0798/voices0798-adelman.shtml 

Bengis, L., et al. (1991). SEEK and college discovery summer programs: Prefreshman, 
English-as-a second language, postfreshman, and science, mathematics & 
technology institutes. 1990 evaluation report. New York, NY: City University of 
New York, Project SEEK. 

Bloom, H. (Ed.). (2005). Learning more from social experiments: Evolving analytic 
approaches. New York, NY: Russell Sage Foundation. 

Deil-Amen, R., & Rosenbaum, J. E. (2003). The social prerequisites of success: Can college 
structure reduce the need for social know-how? The ANNALS of the American 
Academy of Political and Social Science, 586(1), 120-143. 

Levin, H. M., & McEwan, P. J. (2001). Cost-effectiveness analysis: Methods and 
applications. Thousand Oaks, CA: Sage Publications. 

Maher, M.A. (2005). The evolving meaning and influence of cohort membership. 
Innovative Higher Education, 30(3), 195-211. 

Rosenbaum, J. E., Deil-Amen, R., & Person, A. E. (2006). After admission: From college 
access to college success. New York, NY: Russell Sage Foundation. 

Santa Rita, E., & Bacote, J. B. (1997). The benefits of College Discovery Prefreshman 
Summer Program for minority and low-income students. College Student Journal, 
31, 161-173. 

Schochet, P. Z. (2008). Technical methods report: Guidelines for multiple testing in impact 
evaluations (NCEE 2008-4018). Washington, DC: U.S. Department of Education, 
Institute of Education Sciences, National Center for Education Evaluation and 
Regional Assistance. 

Scrivener, S., Bloom, D., LeBlanc, A., Paxson, C., Rouse, C. E., & Sommo, C. (with Au, J., 
Teres, J. J., & Yeh, S.). (2008). A good start: Two-year effects of a freshmen learning 
community program at Kings borough Community College. New York, NY: MDRC. 

Tinto, V. (1997). Classrooms as communities: Exploring the educational character of 
student persistence. Journal of Higher Education, 68(6), 599-623. 



67 




